Recent

Author Topic: Converting a Project from TDbf to SQLite [SOLVED]  (Read 9762 times)

1HuntnMan

  • Sr. Member
  • ****
  • Posts: 349
  • From Delphi 7 to Lazarus
    • NewFound Photo Art
Re: Converting a Project from TDbf to SQLite
« Reply #30 on: February 28, 2025, 05:26:18 pm »
JanRoza,
     Thanks for that bit of knowledge, awesome!

TRon

  • Hero Member
  • *****
  • Posts: 4321
Re: Converting a Project from TDbf to SQLite
« Reply #31 on: February 28, 2025, 06:19:28 pm »
Okay, added the Open but different error, see the attachment after I added the Open statement advised by dseligo.
dseligo also mentioned something else  ;D

It is clearly to do with how the project setup the database components in the designer and how (in which order) things are invoked.

That is why I personally never use the designer to setup/access databases but instead connect everything in code because that way you can not forget to set a single property (and you have to verify/check those for every database component used in a project which literally is a time waster).

So, instead of providing snippets of code and an error-message please provide a full working/compilable example.

Quote
I'll be back in awhile after I create a separate app that just queries by Database for just one table.

It is literally a 5 minute job, see also f.e. this message

And yes, I understand that it is easy to say so if you know howto but there are so many examples provided with Lazarus and FPC not to mention the wiki.

PS: yes, litedac is a commercial product so payware. Never used it so no idea about specifics other than it statically links in sqlite.
« Last Edit: March 01, 2025, 05:28:21 am by TRon »
Today is tomorrow's yesterday.

dseligo

  • Hero Member
  • *****
  • Posts: 1500
Re: Converting a Project from TDbf to SQLite
« Reply #32 on: March 01, 2025, 03:54:03 am »
Okay, added the Open but different error, see the attachment

You mentioned that you are using 'DB Browser'. If you have opened PMSDB.sqlite database in 'DB Browser' it probably won't work simultaneously with you Lazarus/FPC program. You must close 'DB Browser' prior to running your program.

egsuh

  • Hero Member
  • *****
  • Posts: 1594
Re: Converting a Project from TDbf to SQLite
« Reply #33 on: March 01, 2025, 10:11:28 am »
Quote
Okay, added the Open but different error, see the attachment after I added the Open statement advised by dseligo.

I experienced the same difficulty with SQLite3, which would not have happened with Firebird, etc.
I think this is because SQLite is single user DB. The database should not be opened by DB Browser, nor by  Lazarus editor (i.e. at the object inspector) to be opened by "running" application itself.
Even after I close all possible db connection with SQLite3 database, still I met the same error sometimes. I closed Lazarus itself and then re-opened, and then I could do it right.
Not sure exactly what have caused the problem. In most cases I use Firebird.

1HuntnMan

  • Sr. Member
  • ****
  • Posts: 349
  • From Delphi 7 to Lazarus
    • NewFound Photo Art
Re: Converting a Project from TDbf to SQLite
« Reply #34 on: March 01, 2025, 04:02:58 pm »
This morning I just created a simple app with one form to connect to the SQLite3 database and one form with DBNav and a DBGrid.  I don't think I have DB Browser installed??? Also, JanRosa recommended putting the line below in the main form so it's global to the app which good advice. Would this line go into the main form under VAR as a globally defined variable or just under Public? I think I tried that earlier this morning but Laz errors.
Code: Pascal  [Select][+][-]
  1. strDBCntries:= SysUtils.ExtractFilePath(ParamStr(0) + 'Countries.db';
  2.  
From what I understand, I need to add 4 components to the form with the DBNav/DBGrid.  A TSQLite3Connection, TSQLQuery, TSQLTransaction and a DataSource. I think what TRon is recommending from the link is I don't need put these on the form just define them as variables ...
Back to to testing, I'll let you know in awhile, thanks everyone for your great advice???

TRon

  • Hero Member
  • *****
  • Posts: 4321
Re: Converting a Project from TDbf to SQLite
« Reply #35 on: March 01, 2025, 04:17:53 pm »
From what I understand, I need to add 4 components to the form with the DBNav/DBGrid.  A TSQLite3Connection, TSQLQuery, TSQLTransaction and a DataSource. I think what TRon is recommending from the link is I don't need put these on the form just define them as variables ...
You can put those components on a form. The issue is that some of the properties of all these components need to be set. This needs to be done properly and usually in a particular order. My advise would be to not set those properties at design time but rather at runtime.

Depending on the complexity of the program it is usually easier to use a 'third'' unit (usually a datamodule but it can be any unit) on which to place the non-visual database components (at design time). That way every form is able to access that datamodule unit by simply including it in the uses clause. The only thing that is perhaps confusing when using a datamodule is that the datamodule itself has to be created before accessing any components on it.

The post I referred to is setting the  properties of these components manually at runtime which is why I referred to it and not so much about declaring the component variables manually/globally (while it is an option if you prefer to do it that way).

PS: the issue with db related problems is that I (or anyone else) can make up any example for you but it will suffer from the same flaws that you are currently facing. You do not know where to look at/for in order to understand what might be wrong. Therefor the available demo examples probably also are not able to help you out either. You do not share any example code for us to compile and be able to have a look at (don't get me wrong, there are perfect valid reasons not to share current code if you do not want to) but at least you could try and provide a simple example of something that you do not seem to be able to accomplish. There are plenty of example databases (either accompanied with Lazarus or online) that you could refer to in a test example that you would be able to share without compromising any of your existing code-base (or at least any that we are aware of).
« Last Edit: March 01, 2025, 04:43:30 pm by TRon »
Today is tomorrow's yesterday.

JanRoza

  • Hero Member
  • *****
  • Posts: 709
    • http://www.silentwings.nl
Re: Converting a Project from TDbf to SQLite
« Reply #36 on: March 01, 2025, 09:07:46 pm »
Why not put that line in the oncreate method of your main form, that way your databasename is known as soon as your application starts.
Something like:
Code: Pascal  [Select][+][-]
  1. Database.name := SysUtils.ExtractFilePath(ParamStr(0) + 'Countries.db';
  2.  
OS: Windows 11 / Linux Mint 22.1
       Lazarus 4.0 RC FPC 3.2.2
       CodeTyphon 8.70 FPC 3.3.1

egsuh

  • Hero Member
  • *****
  • Posts: 1594
Re: Converting a Project from TDbf to SQLite
« Reply #37 on: March 02, 2025, 04:26:53 am »
Quote
From what I understand, I need to add 4 components to the form with the DBNav/DBGrid.  A TSQLite3Connection, TSQLQuery, TSQLTransaction and a DataSource. I think what TRon is recommending from the link is I don't need put these on the form just define them as variables ...

Important thing is you may have only one connection, i.e. TSQLite3Connection. Others really don't matter. You may have many of these on one form.

1HuntnMan

  • Sr. Member
  • ****
  • Posts: 349
  • From Delphi 7 to Lazarus
    • NewFound Photo Art
Re: Converting a Project from TDbf to SQLite
« Reply #38 on: March 02, 2025, 03:23:10 pm »
Attached is the complete Project for this test app. Just one table in the Database-Countries. It's just an extract from a larger Project that has 21 units. Countries is just one TDbf table for a Drop-Down lookup when updating a Clients, Proposals, Sales Orders, etc. I just took the Countries maintenance unit and made this small test app to learn how to use SQLite3 before tackling the complete project which is a Photographer's Mgt. System (Clients, Contacts, Proposals, Refrences, Sales Orders, Invoicing, Photographer's Mgt., Photos Mgt, etc.
My issue in learning and migrating from TDbf to SQLite3 is not the SQL code but just establish a connection, transaction, query and the Dataset. Can't get past just a small one table Database. I've actually converted all the TDBf tables to SQLite3 tables in a Database.
Thanks anyone if you take this and let me know what I'm doing wrong. Just one query in it to load and view the table in a DBGrid with a Navigator based on the DataSet.

TRon

  • Hero Member
  • *****
  • Posts: 4321
Re: Converting a Project from TDbf to SQLite
« Reply #39 on: March 02, 2025, 03:54:24 pm »
I only had a quick glance (I am heading out the door in a short while):
- code creates TSQLite3Connection. Either the form creates it automatically for you because it is placed on the  form at design time or do not place it on the form at design time and create it manually.
- code assigns name of the Database to DBCEntries (which is a Datasource). It needs to be assigned to DBCntriesConnect.DatabaseName
- The name of the database name on disk is "CNTRIESDB.sqlite3" not "CNTRIESDB"
- Transaction database property not set to DBCntriesConnect.
- Database property of the Query is not set to DBCntriesConnect
- Transaction property of the query is not set to   DBCntriesTrans
- The name of the table inside the database is actually "COUNTRIESDB" not "COUNTRIES" in the query
- query text needs to be set before actually opening the query.

I did not had a good look at the property values that were set at design time but if corrected all the mentions above in code the database/query opens and is able to be browsed by the DB Navigator. Except for one other thing and that is that all the fields in the grid are displayed as "TMEMO" (which is an entirely other issue related to field-type).
« Last Edit: March 02, 2025, 03:59:01 pm by TRon »
Today is tomorrow's yesterday.

TRon

  • Hero Member
  • *****
  • Posts: 4321
Re: Converting a Project from TDbf to SQLite
« Reply #40 on: March 02, 2025, 07:39:28 pm »
I had managed to have a closer looksee  :)

1. seems you manually added some entries in the form type declaration.

HINT: never do that.

If required to manually add components do that in a separate section of the type declaration instead. Either in the public, protected or private sections.

Therefor remove the (commented) entry SQLite3COUNTRIES and the entry SQLTransactionCntries

2. remove the variable declarations DBCntriesConnect, DBCntriesTrans and DBCntriesQry that are situated right beneath the FrmCntriesMgt variable declaration.

Either place the required components on the form (as was already done) or declare them as either fields (in section private, protected or public) or as you did as global variables. Never do both (don't even think about doing it threefold  :P ).

3. Create/add a new private procedure with proto-type "procedure SetupDB;" in the form type declaration, e.g.:

Code: Pascal  [Select][+][-]
  1.   TFrmCntriesMgt = class(TForm)
  2.     BitBtnClose: TBitBtn;
  3.     DSCntries: TDataSource;
  4.     DBGridCntries: TDBGrid;
  5.     DBNavCntries: TDBNavigator;
  6.     PnlTop: TPanel;
  7.     DBCntriesConnect: TSQLite3Connection;
  8.     DBCntriesQry: TSQLQuery;
  9.     DBCntriesTrans: TSQLTransaction;
  10.     procedure BitBtnCloseClick(Sender: TObject);
  11.     procedure FormCreate(Sender: TObject);
  12.     procedure FormShow(Sender: TObject);
  13.   private
  14.     procedure SetupDB;
  15.   public
  16.   end;
  17.  

When added, position the cursor on that new procedure and press ctrl+shift+c. The method should be then automatically be added to the form implementation section.

Add the following code to that implementation so that it reads:

Code: Pascal  [Select][+][-]
  1. procedure TFrmCntriesMgt.SetupDB;
  2. var
  3.   DBName : string = 'CNTRIESDB.sqlite3';
  4. begin
  5.   // References:
  6.   // - https://www.freepascal.org/docs-html/fcl/sqldb/usingsqldb.html
  7.  
  8.   // Step 1: a connection requires a (file)name (and optional credentials).
  9.   // ToDo  : make sure the file exists.
  10.   DBName := SysUtils.ExtractFilePath(ParamStr(0)) + DBName;
  11.   DBCntriesConnect.DatabaseName:= DBName;
  12.  
  13.   // Step 2: A transaction require a connection.
  14.   DBCntriesTrans.DataBase := DBCntriesConnect;
  15.  
  16.   // At this point the database (connection) /can/ be opened (or connected)
  17.   // We open it here just for lolz but it is better to do it after the
  18.   // form was created so that feedback can be provided (f.i when things go wrong)
  19.   DBCntriesConnect.Open;
  20.   // or alternative: DBCntriesConnect.Connected := True;
  21.  
  22.   // Step 3a: Setup Query - a Query requires a database connection
  23.   DBCntriesQry.DataBase := DBCntriesConnect;
  24.  
  25.   // Step 3b: Setup Query - a Query requires a transaction
  26.   DBCntriesQry.Transaction := DBCntriesTrans;
  27.  
  28.   // Step 4: Setup your data sources
  29.   // This can be done at any time it does not specifically has to be step 4.
  30.   // Already (properly) done at design time
  31. end;
  32.  

4. Remove all existing code from the form create event and make a call to procedure SetupDB instead so that it reads:

Code: Pascal  [Select][+][-]
  1. procedure TFrmCntriesMgt.FormCreate(Sender: TObject);
  2. begin
  3.   SetupDB;
  4. end;
  5.  

5. Change the FormShow event implementation so that it reads:

Code: Pascal  [Select][+][-]
  1. procedure TFrmCntriesMgt.FormShow(Sender: TObject);
  2. begin
  3.   DBCntriesQry.SQL.Text:= 'select * from COUNTRIESDB';
  4.   DBCntriesQry.Open;
  5. end;
  6.  

6. Change the BitBtnCloseClick event implementation so that it reads:

Code: Pascal  [Select][+][-]
  1. procedure TFrmCntriesMgt.BitBtnCloseClick(Sender: TObject);
  2. begin
  3.   try
  4.     DBCntriesQry.Close;
  5.   finally
  6.     DBCntriesConnect.Close(true); // alternatively: DBCntries.Connected:= False;
  7.   end;
  8.   FrmCntriesMgt.Close;
  9.   // Please do not shoot yourself in the foot
  10.   // Here the commented code literally removes the ground it stands on
  11.   // FrmCntriesMgt.Free;
  12. end;
  13.  

That should do it. Again I mention, except for the fields showing TMemo in the grid.

PS: Just keep in mind that the sqlite database contains all the tables so that only a single connection and transaction is required (though you could use multiple transactions if wanted). That was also one of the reasons in my previous answer to use global variables (or at least globally accessible) for at least the connection and transaction.
« Last Edit: March 02, 2025, 08:08:50 pm by TRon »
Today is tomorrow's yesterday.

Zvoni

  • Hero Member
  • *****
  • Posts: 2961
Re: Converting a Project from TDbf to SQLite
« Reply #41 on: March 03, 2025, 08:32:09 am »
As to the "opened in DB Browser at the same time" thing:
Someone above mentioned that in "standard"-way you block yourself: That's correct.

I don't remember, if it's enough to use Journal_mode=WAL in DB Browser, or if you need to fire off the Pragma from your Pascal-Code.
Open your Database in DB Browser, switch to Tab "Pragmas", and set Journal-Mode to WAL, then try your code again
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

Thaddy

  • Hero Member
  • *****
  • Posts: 16813
  • Ceterum censeo Trump esse delendam
Re: Converting a Project from TDbf to SQLite
« Reply #42 on: March 03, 2025, 09:06:52 am »
@TRon

Impressed how you demonstrate the pitfalls of using the designtime components and object inspector as opposed to creating the components in code, in understandable language, with clear examples and consistent over multiple answers.

I have never seen it explained more clearly than you did.
Frankly, this is often the only way to use the lower level (connection oriented parts ) database design should be implemented.
Not many people know which properties to set and in what order using the design time components and the object inspector. Creating the components in code will prevent errors.
Changing servers. thaddy.com may be temporary unreachable but restored when the domain name transfer is done.

1HuntnMan

  • Sr. Member
  • ****
  • Posts: 349
  • From Delphi 7 to Lazarus
    • NewFound Photo Art
Re: Converting a Project from TDbf to SQLite
« Reply #43 on: March 03, 2025, 05:29:59 pm »
TRon, thanks loads! Like I was saying earlier, I can write and have been studying SQL as far as coding just having issues just getting started. The only code I am using just for maintaining a COUNTRIES table was 'select * from COUNTRIES'. The rest is just introducing the Database and Connecting. I agree to code it.

I just copied your comments and instructions. I'll be back when I get this one little app. working and can start on converting the whole app from TDbf --> SQLite3. Thanks again!!!

CharlyTango

  • Full Member
  • ***
  • Posts: 112
Re: Converting a Project from TDbf to SQLite
« Reply #44 on: March 04, 2025, 12:12:59 am »
Basically I agree with TRon but...

... this strategy does not solve the problem of providing application-wide database access.

Yes, you can use a separate data access for each form, but this does not work with all databases (be careful with SQLite). There may also be a memory problem on the server when managing many connections, which are not normally necessary.

A complete example for applications can be found here.
It uses a data module and SQLDB components.
This encapsulates the complete database access.

The example uses SQLite, but you can use any SQL database supported by Lazarus. It also provides a logfile to log Data transferred to and from the SQL Server to ease debugging and much much more features useful for database frontend programming..

The form or frame then only contains the actual data components (TSQLQuery, TDataset and, if necessary, data-sensitive controls)
The database connection is assigned to the TSQLQuery in TForm.Activate (because only then are all initialization processes of a form complete) or in a separate procedure in a frame.
I never activate database connections in Lazarus. This only causes problems in the long term if you forget to reset everything.

I only let Lazarus generate the main form automatically. I create all other forms (including the data module) in code. This gives me control over the necessary sequence.

By the way... 'select * from' is more than a bad habit. After 30 years of programming frontends --> Don't ever do that
Lazarus stable, Win32/64

 

TinyPortal © 2005-2018