SQLite save QRCode how to improve speed?

[Int3g3r]

Hello

I write one QR and one Barcode into a SQLite database.
This takes for 600 records around 60s.
Is there way to improve it?

I do it like this (pseudo code):

Code: Pascal  [Select][+][-]

1. procedure GenerateBarcode
2. begin
3.    if qryGrid.RecountCount > 0 then
4.    begin
5.      while not qryGrid.EOF do
6.      begin
7.         GenerateBarcode; //TBitmap
8.         GenerateQRCode; //TBitmap
9.         qryGrid.Edit;
10.         qryGrid.CreateBlobStream(Barcode);
11.         BarcodeStream.SaveToStream(bgStream);
12.         qryGrid.CreateBlobStream(QRCode);
13.         QrCodeStream.SaveToStream(qrStream);
14.         qryGrid.Post;
15.         qryGrid.Next;
16.      end;
17.    end;
18. end;
19.  

Would threading improve this, if so which part should i thread?
Is it possible to edit 100 records at once (in memory), and then post all 100 to the sqlite database, would this improve the speed?
Great would be 1000 records in around 10-20seconds.

Regards Int3g3r

[Thaddy]

The bottleneck is probably the barcode and qrcode generation. What information do you store in it, especially the qrcode, because that can contain *a lot* of info. There is not necessarily anything wrong with your code that I can see except for those two. If you replace these two with dummies it is mere seconds, not minutes. Even milliseconds:600 is not a big number.

Then again: the whole loop is full of things that are possibly computationally expensive by nature.

I would focus on the bar/qrcode generation. And yes, those two can probably be parallelized.
To reassure you: it is not sqlite, which can store 1000ths of records in a second, not a minute, and even on old hardware.

[wp]

Why do you want to store the bar/QRcode in the database directly? These are images, and images in databases is something which should be avoided because they are large and can stuff buffers. You can store the text encoded in the barcode/QRCode directly and create the bar/QRCodes on the fly when needed.

The attached demo creates a table with an integer and a text column for 20,000 records within less than a second. And when you scroll through the database an interleaved-2-of-5 barcode is created instantly from the currently active numeric field (plus checksum in case you are using the current svn version of LazBarCodes), and a QRCode is created from the text field in the same way, just by assigning the field values to the Text property of the bar/QRcode components.

[Zvoni]

Why do you want to store the bar/QRcode in the database directly? These are images, and images in databases is something which should be avoided because they are large and can stuff buffers. You can store the text encoded in the barcode/QRCode directly and create the bar/QRCodes on the fly when needed.

Correct.
Don't store the Image, store the information within the QR/Barcode.
Upside: If you change your QR/Barcode to another Format, the content is (more or less) the same
e.g. Old Barcode Code39 to Code 128 for 1-D or PDF417 to DataMatrix for 2-D.
The Information is still the same, they just look different.

It's the same old discussion with Date-Types:
Never store how something is displayed in the Database, only the Information

[Int3g3r]

Thank you very much for your help and advice.

My biggest problem was that LazReport cannot generate QR codes.
I have also just noticed that the checksum for the 2of5 barcode does not work in LazReport either.

So I decided to save the QR codes as images in the database.
I can then easily read them out later with LazReport.
 
Would there have been a better method?
Would it have been better to save the QR/barcode images on the hard drive and only save the file names in the database?
Which is faster, reading the images from SQLite or from the hard drive?

Regards Int3g3r

[paweld]

So I decided to save the QR codes as images in the database.
I can then easily read them out later with LazReport.

see example in the attachment

[Zvoni]

Thank you very much for your help and advice.

My biggest problem was that LazReport cannot generate QR codes.
I have also just noticed that the checksum for the 2of5 barcode does not work in LazReport either.

So I decided to save the QR codes as images in the database.
I can then easily read them out later with LazReport.
 
Would there have been a better method?
Would it have been better to save the QR/barcode images on the hard drive and only save the file names in the database?
Which is faster, reading the images from SQLite or from the hard drive?

Regards Int3g3r

You have seen that there is a Package called "LazBarcodes" in OPM?

[Int3g3r]

You have seen that there is a Package called "LazBarcodes" in OPM?

Yes, I use LazBarcodes to generate the barcodes.

see example in the attachment

I didn't know that I could transfer the QR barcode via MemoryStream.
I've learned something new, thanks for the example.

Regards Int3g3r

[paweld]

I quoted the wrong part in my previous post.

The example shows how to generate codes on the fly and transfer them to the report - you place a picture object on the report, stet them a name (in the example, “gs1”), and then, in the OnEnterRect event of the report, you load the graphic with the code into the picture object.