Recent

Author Topic: TBufDataset affected IBX?  (Read 3639 times)

incendio

  • Sr. Member
  • ****
  • Posts: 358
Re: TBufDataset affected IBX?
« Reply #15 on: November 16, 2023, 08:08:23 pm »
It is simply just to store temporary data, process this data, then display the results to users.
So... TBufDataset.

I need to sum this column, to do this, must loop the entire records, but looping the entire records on the dataset will triggers lots of methods, such as before post, on calculated, etc.
YIKES. Are you saying you do a count on the client side just to get a sum for a column?

I (now) understand that you are a beginner in database development, and there is nothing wrong with that.
But if you would have stated that from the beginning we could have steered you in a better direction.

Especially if you are doing a SELECT * FROM TABLE and want to display a count/sum, and be responsive with Firebird over a high latency connection like the internet, that's absolutely NOT the way to go. This is really really really inefficient.

With a slow connection what you want to do is not show all 100 or 1000 (or 10.000) records. You only want to retrieve the records that are visible (in a TDBGrid) on screen. And during scrolling, more records are retrieved. That's done automatically. If you need a count or sum then you do a separate SELECT SUM(AMOUNT), COUNT(*) FROM TABLE so you can show the sum and count separately.

You can still do a SELECT * FROM TABLE (although specifying the fields is more efficient) and the TIBQuery will do the caching automatically and only retrieve the needed records. But if you are doing a while not dataset.EOF or a dataset.Last then ALL records are retrieved immediately, resulting in a MAJOR slowdown, and you don't want that.

To avoid trigger those methods & events, I copied values of that column to temporary table and do same processing, then display the result in a formated value, 1000 will be displayed as 1,000.
Even with that method... ALL the records need to be retrieved. You don't do that in a database program for the reason stated above.

Above method does take some getting use to but it's much better than using a virtual database or doing a loop through ALL the records.
(if you have 1000 or more records, your method would become unbearable slow over the internet, even with a memory table.)

I hope that the explanation above is clear otherwise feel free to ask questions.
Data is from users input, so it is not in the table yet.

Just imagine input a store sales in POS application, where there are price & qty that need to be sum.

Total Price & qty, displayed to users in a formated value. At this point, nothing to do with the server!

rvk

  • Hero Member
  • *****
  • Posts: 6886
Re: TBufDataset affected IBX?
« Reply #16 on: November 16, 2023, 08:23:53 pm »
Data is from users input, so it is not in the table yet.

Just imagine input a store sales in POS application, where there are price & qty that need to be sum.

Total Price & qty, displayed to users in a formated value. At this point, nothing to do with the server!
In that case you also don't work with IBX. You use a TBufDataset.
And hope the POS doesn't crash or switch of and you loose all data  ;)

And then when the POS has a connection you can transfer that over to the server database.

BTW. I think most POS do have a (slow) connection (to the cloud) and in that case you work with a middleware method.
Not a direct connection to the database server but via a middleman (for example via API/json or something similar) and that piece of software communicates with the server. That way you don't have to transfer a lot over a slow connection.

CharlyTango

  • Full Member
  • ***
  • Posts: 169
Re: TBufDataset affected IBX?
« Reply #17 on: November 17, 2023, 09:26:19 am »
Data is from users input, so it is not in the table yet.

Just imagine input a store sales in POS application, where there are price & qty that need to be sum.

Total Price & qty, displayed to users in a formated value. At this point, nothing to do with the server!

Preface: I don't know how IBX components work exactly.

That's why CachedUpdates entered the game.
A Dataset (which in your case is the IBX query component) is usually capable of manipulating data locally. In your POS example that would be inserting deleting and editing data until everything is fine. After everything is done ApplyUpdates will transfer (inserted, changed, deleted) data to the database server.
Lazarus stable, Win32/64

tonyw

  • Sr. Member
  • ****
  • Posts: 344
    • MWA Software
Re: TBufDataset affected IBX?
« Reply #18 on: November 17, 2023, 10:13:21 am »
Data is from users input, so it is not in the table yet.

Just imagine input a store sales in POS application, where there are price & qty that need to be sum.

Total Price & qty, displayed to users in a formated value. At this point, nothing to do with the server!

Preface: I don't know how IBX components work exactly.

That's why CachedUpdates entered the game.
A Dataset (which in your case is the IBX query component) is usually capable of manipulating data locally. In your POS example that would be inserting deleting and editing data until everything is fine. After everything is done ApplyUpdates will transfer (inserted, changed, deleted) data to the database server.
ApplyUpdates is really just a way of delaying the write through to the database. It came from the original Borland IBX and I am not convinced it is that useful in the modern world. The time taken to write through to the database should not be significant, especially if the Firebird Server is configured for lazy writes. Firebird transaction control is a much better way of rolling back any changes if a transaction needs to be canceled. It also co-ordinates multiple clients operating on the same database. If you want your POS to have more than one terminal then you really need to go down this path and avoid Cached updates.

incendio

  • Sr. Member
  • ****
  • Posts: 358
Re: TBufDataset affected IBX?
« Reply #19 on: November 17, 2023, 11:20:04 am »
Data is from users input, so it is not in the table yet.

Just imagine input a store sales in POS application, where there are price & qty that need to be sum.

Total Price & qty, displayed to users in a formated value. At this point, nothing to do with the server!

Preface: I don't know how IBX components work exactly.

That's why CachedUpdates entered the game.
A Dataset (which in your case is the IBX query component) is usually capable of manipulating data locally. In your POS example that would be inserting deleting and editing data until everything is fine. After everything is done ApplyUpdates will transfer (inserted, changed, deleted) data to the database server.
ApplyUpdates is really just a way of delaying the write through to the database. It came from the original Borland IBX and I am not convinced it is that useful in the modern world. The time taken to write through to the database should not be significant, especially if the Firebird Server is configured for lazy writes. Firebird transaction control is a much better way of rolling back any changes if a transaction needs to be canceled. It also co-ordinates multiple clients operating on the same database. If you want your POS to have more than one terminal then you really need to go down this path and avoid Cached updates.
Does it mean don't use ApplyUpdates/CancelUpdates, but use Post method within transaction, and when users click Save/Cancel button,  commit or rollback transaction?

tonyw

  • Sr. Member
  • ****
  • Posts: 344
    • MWA Software
Re: TBufDataset affected IBX?
« Reply #20 on: November 17, 2023, 05:21:16 pm »
Yes

 

TinyPortal © 2005-2018