sqlitelaz.lpk linking error

[TRon]

Don't be so sure it is not my fault. Quite possibly I may have done something I should not have in the name of learning, gaining more knowledge and wisdom.

Well right now with the mention of your latest issue I am indeed very unsure what it is you ran into.

fwiw: To make sure, I've installed sqlite3laz package in/with Lazarus 2.2.4 without any issues.

I have multiple linux kernels and multiple versions of Lazarus on this box 

Just make sure to separate the configuration (directory) between different Lazarus versions

edit: try to install package sqlite3laz instead

I tried and I am getting a weird error. I have attached a screenshot.

I am not familiar with the issue as presented.

Which version (please be exact) of the Lazarus IDE do you use ?

Version 2.2.5. Screenshot attached

Thank you. That allowed me to use/test 2.2.4 and install sqlite3laz and see if that worked which it did. Therefor atm I can only assume that you ran into another (unrelated) issue.

[Aruna]

I am not familiar with the issue as presented.

I was spending too much time trying to get the SQLDB tab enabled. So I decided to shelf this for a later time and look for alternative options. I dropped a DataSource, CSVDataset and a TDBGrid on a form and it took me a while to figure out things but now I have a working tool. I have attached the source for anyone interested. When I type into the Edit box and click the button the grid updates and displays the dataset correctly.

If you click the column headers it will sort the entire dataset. Sorting is limited to ascending, is there a way to sort in other ways? I also want to implement a functionality when I type the dataset should update and display any records that match what I have typed so far. What would be the best way to do this?

Thank you all for your advice and help. The source is below:

Code: Pascal  [Select][+][-]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.   Classes, SysUtils, db, csvdataset, Forms, Controls, Graphics, Dialogs,
9.   DBGrids, ExtDlgs, StdCtrls, FileUtil ;
10.  
11. type
12.  
13.   { TForm1 }
14.  
15.   TForm1 = class(TForm)
16.     Button1: TButton;
17.  
18.     CSVDataset1: TCSVDataset;
19.     DataSource1: TDataSource;
20.     DBGrid1: TDBGrid;
21.     OpenPictureDialog1: TOpenPictureDialog;
22.     SearchEdit: TEdit;
23.     SearchButton: TButton;
24.  
25.     procedure DBGrid1TitleClick(Column: TColumn);
26.     procedure FormCreate(Sender: TObject);
27.     procedure SearchButtonClick(Sender: TObject);
28.  
29.   private
30.  
31.   public
32.  
33.   end;
34.  
35. var
36.   Form1: TForm1;
37.  
38.  
39. implementation
40.  
41. {$R *.lfm}
42.  
43. { TForm1 }
44.  
45. procedure TForm1.DBGrid1TitleClick(Column: TColumn);
46. begin
47.   CSVDataset1.IndexFieldNames := Uppercase(Column.Title.Caption);
48. end;
49.  
50. procedure TForm1.FormCreate(Sender: TObject);
51. begin
52.   CSVDataset1.LoadFromCSVFile('/home/aruna/qemu/worldcities.csv');
53.   CSVDataset1.Open;
54. end;
55.  
56. procedure TForm1.SearchButtonClick(Sender: TObject);
57. var
58.   country: string;
59. begin
60.   country := SearchEdit.Text;
61.   CSVDataSet1.Filtered := False; // Disable previous filters
62.   CSVDataSet1.Filter := Format('country = ''%s''', [country]);
63. // CSVDataSet1.Filter := Format('country = '' LIKE ''%s''', [country]);
64.   CSVDataSet1.Filtered := True;
65.   end;
66.  
67.  
68. end.

The csv data is too large to attach so I have truncated and given below:

Code: Pascal  [Select][+][-]

1. city,city_ascii,lat,lng,country,iso2,iso3,admin_name,capital,population,id
2. Tokyo,Tokyo,35.6897,139.6922,Japan,JP,JPN,Tōkyō,primary,37732000,1392685764
3. Jakarta,Jakarta,-6.1750,106.8275,Indonesia,ID,IDN,Jakarta,primary,33756000,1360771077
4. Delhi,Delhi,28.6100,77.2300,India,IN,IND,Delhi,admin,32226000,1356872604
5. Guangzhou,Guangzhou,23.1300,113.2600,China,CN,CHN,Guangdong,admin,26940000,1156237133
6. Mumbai,Mumbai,19.0761,72.8775,India,IN,IND,Mahārāshtra,admin,24973000,1356226629
7. Manila,Manila,14.5958,120.9772,Philippines,PH,PHL,Manila,primary,24922000,1608618140
8. Shanghai,Shanghai,31.1667,121.4667,China,CN,CHN,Shanghai,admin,24073000,1156073548
9. São Paulo,Sao Paulo,-23.5500,-46.6333,Brazil,BR,BRA,São Paulo,admin,23086000,1076532519
10. Seoul,Seoul,37.5600,126.9900,South Korea,KR,KOR,Seoul,primary,23016000,1410836482
11. Mexico City,Mexico City,19.4333,-99.1333,Mexico,MX,MEX,Ciudad de México,primary,21804000,1484247881
12. Cairo,Cairo,30.0444,31.2358,Egypt,EG,EGY,Al Qāhirah,primary,20296000,1818253931
13. New York,New York,40.6943,-73.9249,United States,US,USA,New York,,18972871,1840034016
14. Dhaka,Dhaka,23.7639,90.3889,Bangladesh,BD,BGD,Dhaka,primary,18627000,1050529279
15. Beijing,Beijing,39.9040,116.4075,China,CN,CHN,Beijing,primary,18522000,1156228865
16. Kolkāta,Kolkata,22.5675,88.3700,India,IN,IND,West Bengal,admin,18502000,1356060520
17. Bangkok,Bangkok,13.7525,100.4942,Thailand,TH,THA,Krung Thep Maha Nakhon,primary,18007000,1764068610
18. Shenzhen,Shenzhen,22.5350,114.0540,China,CN,CHN,Guangdong,minor,17619000,1156158707
19. Moscow,Moscow,55.7558,37.6178,Russia,RU,RUS,Moskva,primary,17332000,1643318494
20. Buenos Aires,Buenos Aires,-34.5997,-58.3819,Argentina,AR,ARG,Buenos Aires, Ciudad Autónoma de,primary,16710000,1032717330
21. Lagos,Lagos,6.4550,3.3841,Nigeria,NG,NGA,Lagos,minor,16637000,1566593751
22. Istanbul,Istanbul,41.0136,28.9550,Turkey,TR,TUR,İstanbul,admin,16079000,1792756324
23. Karachi,Karachi,24.8600,67.0100,Pakistan,PK,PAK,Sindh,admin,15738000,1586129469
24. Bangalore,Bangalore,12.9789,77.5917,India,IN,IND,Karnātaka,admin,15386000,1356410365
25. Ho Chi Minh City,Ho Chi Minh City,10.7756,106.7019,Vietnam,VN,VNM,Hồ Chí Minh,admin,15136000,1704774326
26. Ōsaka,Osaka,34.6939,135.5022,Japan,JP,JPN,Ōsaka,admin,15126000,1392419823
27. Chengdu,Chengdu,30.6600,104.0633,China,CN,CHN,Sichuan,admin,14645000,1156421555
28. Tehran,Tehran,35.6892,51.3889,Iran,IR,IRN,Tehrān,primary,14148000,1364305026
29. Kinshasa,Kinshasa,-4.3250,15.3222,Congo (Kinshasa),CD,COD,Kinshasa,primary,12836000,1180000363
30. Rio de Janeiro,Rio de Janeiro,-22.9111,-43.2056,Brazil,BR,BRA,Rio de Janeiro,admin,12592000,1076887657
31. Chennai,Chennai,13.0825,80.2750,India,IN,IND,Tamil Nādu,admin,12395000,1356374944
32.  

[TRon]

I was spending too much time trying to get the SQLDB tab enabled.

I understand you moved on but just a few remarks.

The last error you showed is a unit that should only be compiled when compiling for mac and it really beats me how you got there.

afaik sqlite3laz 0.4 (note, not sqlitelaz 0.4) only adds the sqlite dataset to the data access tab.

In order to get the SQLDB tab you are required to install SQLDBLaz 1.0.2 (see also wiki)

Both can be installed from the install/uninstall packages menu-item in the IDE (move from right to left listbox) and then press the save and rebuild IDE button. If you installed the bigide then the SQLDB package is already installed by default.

So I decided to shelf this for a later time and look for alternative options.

That is some nice progress.

Sorting is limited to ascending, is there a way to sort in other ways?

For a CVSDataSet, not that I am aware of.

I also want to implement a functionality when I type the dataset should update and display any records that match what I have typed so far. What would be the best way to do this?

You already implemented that by using the filter ?

Or are you specifically asking about how to (re)act on keypresses in TEdit ? (see also TEdit doc)

In case you are asking about TEdit then have a look at the documentation I linked to regarding the events onkeydown, onkeypress and onkeyup. Everytime a key is accepted you could invoke setting/updating your filter.

Do realize though that if your dataset is huge that there can be a delay. You can try approach that with using a thread (for filtering) or a timer (for pressing keys) or perhaps someone else has a better idea ?

[Aruna]

The last error you showed is a unit that should only be compiled when compiling for mac and it really beats me how you got there.

If you look again at the screenshot I posted regarding this error you will see a file path ending with appkit.pas I think I was trying to create a new graphical app but I also have no clue what I did to include this in the project. It was an error caused by my experimenting  nothing to do with Lazarus 

afaik sqlite3laz 0.4 (note, not sqlitelaz 0.4) only adds the sqlite dataset to the data access tab.
In order to get the SQLDB tab you are required to install SQLDBLaz 1.0.2 (see also wiki)

Both can be installed from the install/uninstall packages menu-item in the IDE (move from right to left listbox) and then press the save and rebuild IDE button. If you installed the bigide then the SQLDB package is already installed by default.

This is what I wanted to know. Thank you very much. You have effectively cleared up the question I had. I had read about bigide but so far have not compiled and used it.

So I decided to shelf this for a later time and look for alternative options.

That is some nice progress.

Thank you. Yes it is nice when your code does what you want it to..

Sorting is limited to ascending, is there a way to sort in other ways?

For a CVSDataSet, not that I am aware of.

If I wanted to try and build that functionality into a CVSDataset where do I find the source code? And are there any tutorials, guides, or documentation about how to go about doing this?

I also want to implement a functionality when I type the dataset should update and display any records that match what I have typed so far. What would be the best way to do this?

You already implemented that by using the filter ?

Yes I implemented a filter and it works but not each key press real time dataset refresh and update (yet)

Or are you specifically asking about how to (re)act on keypresses in TEdit ? (see also TEdit doc)

In case you are asking about TEdit then have a look at the documentation I linked to regarding the events onkeydown, onkeypress and onkeyup. Everytime a key is accepted you could invoke setting/updating your filter.

I finally got it working but I do not much like the way I am doing things. It was the only way I could think of and it does work so... we have realtime every keypress CSVdataset updates. The source is below.

Code: Pascal  [Select][+][-]

1. procedure TForm1.EditFilterChange(Sender: TObject);
2. var
3.   country: string;
4. begin
5.   country := EditFilter.Text;
6.   country := country+'*';  //THE TRICK WAS TO ADD A WILDCARD "*" TO THE FILTER STRING AT THE END OF EACH KEYPRESS. THERE HAS TO BE A BETTER WAY?
7.   CSVDataSet1.Filtered := False; // Disable previous filters
8.   CSVDataSet1.Filter := Format('country = ''%s''', [country]);
9.   CSVDataSet1.Filtered := True;
10.   end;                      

Do realize though that if your dataset is huge that there can be a delay. You can try approach that with using a thread (for filtering) or a timer (for pressing keys) or perhaps someone else has a better idea ?

You know the funny thing is my dataset is not big data as such but it is a large set of data comprising of 44,692 records. So far the response has been very fast. Almost instant I feel. I threw everything at it trying to break things or get it to slow down on screen updates and refresh and so far Lazarus has handled it extremely well. Once again many thanks for the bigide tip and all the other insights.

[TRon]

If I wanted to try and build that functionality into a CVSDataset where do I find the source code? And are there any tutorials, guides, or documentation about how to go about doing this?

The normal route to determine where the source-code resides is to open the help and search for the components name. Unfortunately your version of Lazarus' accompanied helpfiles do not seem to include the specific help-page (by default).

Alternatively it is possible to make a simple search of the component's name in a search-engine (TCSVDataSet and FreePascal) that lands us here and which tells us that the component is part of package fcl-db (top right corner) and resides in unit csvdataset (top left corner).

Normally the Lazarus installation comes accompanied with the source-code for the different FPC packages (because the IDE relies on them) so you can access the source-code that way.

Alternatively you can find the source-code of the package fcl-db here and for csvdataset in particular here.

Do note that the revision of that online source-code is trunk/main and which does not have to match 100% with the code accompanied with your version of lazarus (2.2.5).

However realize that csv is a pretty restricted format and that normally the underlying db engine is responsible for the ordering of data unless you have the data in the db stored in different orders (keys). I do not even know if CSVDataset is able to select different keys.

SQL for example allows for change of the sortorder of a fieldvalue using a select statement.

I am not aware of any documentation regarding adjusting existing code of a csvdataset (or datasets in general) to customize it. You really would have to look the source-code and assess if it is even possible to accomplish what you have set out to make it do.

I finally got it working but I do not much like the way I am doing things.

Ah, ic you used the onchange event. That is even better :-)

What is it exactly that you do not like about that code ?  The filter itself using a mask ? Because there really isn't a better way to match contents of a field when using a filter.

You know the funny thing is my dataset is not big data as such but it is a large set of data comprising of 44,692 records. So far the response has been very fast. Almost instant I feel

If you are happy with the reaction time then you do not have to improve any further. I was referring to millions of records and even then things depend on the speed of your hardware 

[Aruna]

However realize that csv is a pretty restricted format and that normally the underlying db engine is responsible for the ordering of data unless you have the data in the db stored in different orders (keys). I do not even know if CSVDataset is able to select different keys.

Lazarus does allow a CSVDataset to be treated like a database. And yes it can be indexed using code shown below:

Code: Pascal  [Select][+][-]

1. // Index by field names (Name, Position, Salary)
2.   CsvDataset.IndexFieldNames('Name,Position,Salary');

I am not aware of any documentation regarding adjusting existing code of a csvdataset (or datasets in general) to customize it. You really would have to look the source-code and assess if it is even possible to accomplish what you have set out to make it do.

I just found you can do something like this using DESC short for descending which you obviously knew  :

Code: Pascal  [Select][+][-]

1. LoadFromCSVFile('worldcities.csv');
2. IndexFieldNames := 'country DESC;population DESC;

I finally got it working but I do not much like the way I am doing things.

Ah, ic you used the onchange event. That is even better :-)

Yes it works well.

What is it exactly that you do not like about that code ?  The filter itself using a mask ? Because there really isn't a better way to match contents of a field when using a filter.

Yes the mask I felt was a lame attempt by me after trying many other attempts. But if you say that is the way to do it I will stop feeling there must be another better way 

You know the funny thing is my dataset is not big data as such but it is a large set of data comprising of 44,692 records. So far the response has been very fast. Almost instant I feel

If you are happy with the reaction time then you do not have to improve any further. I was referring to millions of records and even then things depend on the speed of your hardware 

I plugged in a CSV dataset that has over a million records. 5 million. And you were right the delay is very noticeable and annoying. Anything upto fifty thousand records Lazarus handles beautifully. As the data gets larger the update/refresh becomes slower. In my case it took almost 20 something seconds.It does work but who wants to wait 20 seconds in between each keypress

[Aruna]

The last error you showed is a unit that should only be compiled when compiling for mac and it really beats me how you got there.

Hi, I just found out how I managed to get the unit for mac in my project. I was trying to install customdrawn0.0 and when I asked Lazarus to install then re-build the IDE "that" is when somehow the mac unit get's pulled in. Just thought I will let you know.

[TRon]

@aruna:
Thank you for letting us know. That must be one of the most weird things I have seen mentioned in ages. If you are able to reproduce and have the time then by all means make a specific mention of it (error, log, steps to reproduce).

There are issues with recompiling the IDE when system installed (access rights) so your issue might be a specific symptom but not that that I can recall to have seen before.

[Aruna]

@aruna:
Thank you for letting us know. That must be one of the most weird things I have seen mentioned in ages. If you are able to reproduce and have the time then by all means make a specific mention of it (error, log, steps to reproduce).

I will do so..

There are issues with recompiling the IDE when system installed (access rights) so your issue might be a specific symptom but not that that I can recall to have seen before.

I uploaded a screen recoding of mydesktop here which clearly shows the weird behaviour.

[TRon]

I uploaded a screen recoding of mydesktop here which clearly shows the weird behaviour.

Thank you and IC.


Strange as it works for me ^tm (just tested with Lazarus 3.4)

Did you by any chance made any changes to tools, configure "Build Lazarus" ?

[Aruna]

Strange as it works for me ^tm (just tested with Lazarus 3.4)

I am actually thinking of installing the latest Lazarus. I think I have two versions of Lazarus installed on my system. Should I backup them first then remove/delete everything or just download the precompiled binary then extract and run the intsaller?

Did you by any chance made any changes to tools, configure "Build Lazarus" ?

I am attaching a screenshot. As far as I can remember, I do not think I made any changes. 

[TRon]

I am actually thinking of installing the latest Lazarus.

Should be working with previous versions of Lazarus as well, though no experience with installing that package in particular.

I think I have two versions of Lazarus installed on my system. Should I backup them first then remove/delete everything or just download the precompiled binary then extract and run the intsaller?

Depends on how those Lazarus' were installed

I would advise against running the installer, at least not using the default location for installation in case you have (or want to have) multiple versions on your system.

The part where you have to pay attention with multiple installations of Lazarus is the Lazarus configuration directory (see also https://wiki.freepascal.org/Multiple_Lazarus) and ofc you can't have all Lazarus versions installed in the exact same directory (f.e. I have them separated by version number, e.g. /apps/lazarus/2.2.4/..., /apps/lazarus/3.4/... etc).

As long as you keep the configuration directories separated then there is no need to backup (your config directory) (but make sure you do just in case). There is also a process recently described by I believe user dbannon on the forum that describes how to upgrade your Lazarus from a previous version without the need to install everything again manually (I would have to look that up in case I am able to find it and in case you have an interest in that) (*)

I personally build my Lazarus' from source (see also https://wiki.lazarus.freepascal.org/Getting_Lazarus#Compiling_and_running_Lazarus) and have them installed in user space (e.g. make users part of a custom dev group with corresponding access rights so that rebuilding the IDE does not run into issues = known Achilles' heel atm for some platforms).

I am attaching a screenshot. As far as I can remember, I do not think I made any changes.

Thank you. Indeed that looks all pretty normal.

edit: (*) see https://forum.lazarus.freepascal.org/index.php/topic,67859.msg523279.html#msg523279

[Aruna]

I am actually thinking of installing the latest Lazarus.

Should be working with previous versions of Lazarus as well, though no experience with installing that package in particular.

I have been experimenting so much since I first installed the Lazarus IDE and may have introduced unintended errors along the way due to being a newbie. The reason I was thinking to re-install everything from scratch is to try and get back to a clean error free install.

I think I have two versions of Lazarus installed on my system. Should I backup them first then remove/delete everything or just download the precompiled binary then extract and run the intsaller?

Depends on how those Lazarus' were installed

Yes, very true.

I would advise against running the installer, at least not using the default location for installation in case you have (or want to have) multiple versions on your system.

Alright understood and will be careful.

The part where you have to pay attention with multiple installations of Lazarus is the Lazarus configuration directory (see also https://wiki.freepascal.org/Multiple_Lazarus) and ofc you can't have all Lazarus versions installed in the exact same directory (f.e. I have them separated by version number, e.g. /apps/lazarus/2.2.4/..., /apps/lazarus/3.4/... etc).

Thank you, I will go through the wiki article.

As long as you keep the configuration directories separated then there is no need to backup (your config directory) (but make sure you do just in case). There is also a process recently described by I believe user dbannon on the forum that describes how to upgrade your Lazarus from a previous version without the need to install everything again manually (I would have to look that up in case I am able to find it and in case you have an interest in that) (*)

Yes I would be very interested in how dbannon upgraded his Lazarus without installing everything manually.

I personally build my Lazarus' from source (see also https://wiki.lazarus.freepascal.org/Getting_Lazarus#Compiling_and_running_Lazarus) and have them installed in user space (e.g. make users part of a custom dev group with corresponding access rights so that rebuilding the IDE does not run into issues = known Achilles' heel atm for some platforms).

Good to know. Thank you again!

[TRon]

Yes I would be very interested in how dbannon upgraded his Lazarus without installing everything manually.

Just to make sure you did not miss it, I edited my post and added the url right beneath the horizontal line (sometimes that gets mistaken for a tagline)

[Aruna]

Yes I would be very interested in how dbannon upgraded his Lazarus without installing everything manually.

Just to make sure you did not miss it, I edited my post and added the url right beneath the horizontal line (sometimes that gets mistaken for a tagline)

Thank you. I did miss it thinking it was a tagline. I did wonder why is there a edit (*) in there but never gave it more thought