Recent

Author Topic: Smart way to get changes on Db table.  (Read 1810 times)

BSaidus

  • Hero Member
  • *****
  • Posts: 529
  • lazarus 1.8.4 Win8.1 / cross FreeBSD
Smart way to get changes on Db table.
« on: February 12, 2024, 04:41:52 pm »
Hello.
Let say that I have an sqlite3 table
Code: Pascal  [Select][+][-]
  1. t1 (id integer, name varchar(20), address varchar(50) )
  2.  
Is there any smart way to know changed records after posting data ?
I mean :
Code: Pascal  [Select][+][-]
  1.   // let record 1 has a values
  2.  
  3.      id          = 1,
  4.      name     = 'toto',
  5.      address  = '33 av. lebhayer'
  6.  
I changed the name value to
Code: Pascal  [Select][+][-]
  1.   // let record 1 has a values
  2.  
  3.      id          = 1,
  4.      name     = 'tata',
  5.      address  = '33 av. lebhayer'
  6.  
after commuting changes, Is there any way to know that only the name field changed.

For now, I proceed like this.
  when creating form, I get the initial value before any changes to variable,
then after omitting changes I compare current values with the first ones I get in the FormCreate.

Code: Pascal  [Select][+][-]
  1.    // get_Initial_values
  2.    // POST changes
  3.    // get_Current_value >> Diff >> get_Initial_values
  4.  
  5.  

Thank you

 
lazarus 1.8.4 Win8.1 / cross FreeBSD
dhukmucmur vernadh!

cdbc

  • Hero Member
  • *****
  • Posts: 952
    • http://www.cdbc.dk
Re: Smart way to get changes on Db table.
« Reply #1 on: February 12, 2024, 06:33:12 pm »
Hi
I think SQLite has something called 'rows_affected' tucked away somewhere deeper then the dataset, maybe in TSQLite3Connection?!?
Happy hunting
Regards Benny
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Smart way to get changes on Db table.
« Reply #2 on: February 12, 2024, 07:07:44 pm »
What cdbc means, is function TSQLQuery.RowsAffected, if that is, what you was looking for.

TRon

  • Hero Member
  • *****
  • Posts: 2219
Re: Smart way to get changes on Db table.
« Reply #3 on: February 13, 2024, 03:56:50 am »
SQLite seem to support a so called snapshot  (when support for it was compiled in) which can be used to differentiate between two points in time. I do not have any experience with SQLite and snapshots though so can't provide practical example.

Thaddy

  • Hero Member
  • *****
  • Posts: 13961
  • Probably until I exterminate Putin.
Re: Smart way to get changes on Db table.
« Reply #4 on: February 13, 2024, 08:10:47 am »
SQLite seem to support a so called snapshot  (when support for it was compiled in) which can be used to differentiate between two points in time. I do not have any experience with SQLite and snapshots though so can't provide practical example.
That is an interesting feature that I did not know about in SQLite.
I found a good example on the web.
Suppose we have two tables, student and new_student. Then we can create a snapshot like so:
Code: SQL  [Select][+][-]
  1. CREATE SNAPSHOT snapshot1 AS
  2. SELECT student.rollno, student.name FROM student
  3. UNION ALL
  4. SELECT new_student.rollno, new_student.name FROM new_student;

Now suppose we have created two snapshots over time, we can query the difference like so:
Code: SQL  [Select][+][-]
  1. SELECT * FROM snapshot1 EXCEPT SELECT * FROM snapshot2;
That was really easy. I will use it more often. Tnx for the tip!
« Last Edit: February 13, 2024, 09:18:03 am by Thaddy »
Specialize a type, not a var.

TRon

  • Hero Member
  • *****
  • Posts: 2219
Re: Smart way to get changes on Db table.
« Reply #5 on: February 13, 2024, 08:26:28 am »
Thank you for providing an easy to understand example Thaddy

rvk

  • Hero Member
  • *****
  • Posts: 5946
Re: Smart way to get changes on Db table.
« Reply #6 on: February 13, 2024, 08:30:44 am »
after commuting changes, Is there any way to know that only the name field changed.
What the other seem to miss about this question is that you want the changed FIELDS. Not if there are any changed records.

No, there is no way to get that from SQL itself.
The only way is to see if the field is changed before posting by looking at something like TDBEdit.Modified.
But you would need to do that at the right time.

TRon

  • Hero Member
  • *****
  • Posts: 2219
Re: Smart way to get changes on Db table.
« Reply #7 on: February 13, 2024, 08:51:48 am »
No, there is no way to get that from SQL itself.
Well, in case a snapshot is not helpful and with a bit of imagination you could end up with something like this.

Overkill perhaps but imho so is the original question .....

Quote
But you would need to do that at the right time.
.. because if you do it at the right time (and thus avoid making things more complicated than they already are) then you would either take your approach or know beforehand what you are going to change. A simple check before posting is enough to set a state. And my SQL is a bit rusty but isn't it possible to manage that with a trigger ?

rvk

  • Hero Member
  • *****
  • Posts: 5946
Re: Smart way to get changes on Db table.
« Reply #8 on: February 13, 2024, 08:58:53 am »
No, there is no way to get that from SQL itself.
Well, in case a snapshot is not helpful and with a bit of imagination you could end up with something like this.
This still doesn't answer the original question.
What is an easy way to know which FIELD was changed.

Given link will give you the multiple versions of the complete record but still doesn't tell you which FIELD was changed.

There really isn't any other way than to check all the values.
(you could loop the fields but you still need to do that through code, there is no magic this_field_changed property.)

TRon

  • Hero Member
  • *****
  • Posts: 2219
Re: Smart way to get changes on Db table.
« Reply #9 on: February 13, 2024, 09:08:25 am »
This still doesn't answer the original question.
What is an easy way to know which FIELD was changed.

Given link will give you the multiple versions of the complete record but still doesn't tell you which FIELD was changed.
Pardon me in case I have it wrong but isn't name of column that was altered not the same as "which field"

Quote
An alternative approach is to store null for any column that didn’t change since the previous version. This saves on space, but introduces a new challenge: what if the user updated a column and set the new value to null? That change would be indistinguishable from no change at all.

My solution then is to use this _mask column. Every column in the table gets a power-of-two number—1, 2, 4, 8 for id, name, age and weight respectively.

The _mask then records the sum of those numbers as a bitmask. In this way, the _history row need only store information for columns that have changed, with an overhead of just four extra integer columns to record the metadata about that change.

Besides that when you have two snapshots isn't it then possible to compare them in order to create a diff (of returning only the field-names that where changed) using SQL only ?

I'll get a closer look at it myself but have to do it (much) later as I have other things on my plate right now.

Quote
There really isn't any other way than to check all the values.
No argument from me there, especially not with regards to the link I posted and from which I quoted.
« Last Edit: February 13, 2024, 09:12:29 am by TRon »

rvk

  • Hero Member
  • *****
  • Posts: 5946
Re: Smart way to get changes on Db table.
« Reply #10 on: February 13, 2024, 09:15:46 am »
This still doesn't answer the original question.
What is an easy way to know which FIELD was changed.

Given link will give you the multiple versions of the complete record but still doesn't tell you which FIELD was changed.
Pardon me in case I have it wrong but isn't name of column that was altered not the same as "which field"
Yes, you are correct there. The _mask field is the one that is wanted.
But the question is essentially how do you calculate that _mask field in an easy ("smart") way.

The question wasn't even to store that record with _mask... but just to know which field is changed (so what the _mask would be).

SQLite doesn't calculate that _mask field itself... you need to calculate it yourself and that's what TS is asking.
(the diff method was already found by TS: "// get_Current_value >> Diff >> get_Initial_values" )

TRon

  • Hero Member
  • *****
  • Posts: 2219
Re: Smart way to get changes on Db table.
« Reply #11 on: February 13, 2024, 09:27:35 am »
But the question is essentially how do you calculate that _mask field in an easy ("smart") way.
Ok ok... well I imagined the suggestions made where at least a bit smartER. But fair enough  :)

Quote
The question wasn't even to store that record with _mask... but just to know which field is changed (so what the _mask would be).
No indeed, that is true. Knowing beforehand is (still) the better approach.

I stumbled upon this thread which essentially (the last answer) uses the same trigger mechanism but the first answer might perhaps be (more) interesting ? (although it would again boil down to a comparison)

Zvoni

  • Hero Member
  • *****
  • Posts: 2218
Re: Smart way to get changes on Db table.
« Reply #12 on: February 13, 2024, 11:03:28 am »
Right, to get to the Bottom of this.
rvk has it right: There is no auto-magic to tell you which field has changed.
What TS is after is called "auditing", and has to be implemented by the developer him/herself.
SQLite (or any other DBMS i 've worked with) is not going to do it for him/her

usually done with an auditing-table which gets filled AFTER INSERT, BEFORE/AFTER UPDATE and BEFORE DELETE (Now there are three hints how to do that).

NotaBene: Auditing tables are usually done with using EAV Antipattern
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

Thaddy

  • Hero Member
  • *****
  • Posts: 13961
  • Probably until I exterminate Putin.
Re: Smart way to get changes on Db table.
« Reply #13 on: February 13, 2024, 11:05:47 am »
Of course you can do that since in SQLite you can query the meta data and take snapshots from that. And I guess that works. SQLite <> IB
Specialize a type, not a var.

Zvoni

  • Hero Member
  • *****
  • Posts: 2218
Re: Smart way to get changes on Db table.
« Reply #14 on: February 13, 2024, 11:09:19 am »
Of course you can do that since in SQLite you can query the meta data and take snapshots from that. And I guess that works. SQLite <> IB
Which meta_data would that be?

Since TRon quoted the SQLite-Documentation, then people should actually read it, because there is a very important detail:
https://sqlite.org/c3ref/snapshot.html
Quote
An instance of the snapshot object records the state of a WAL mode database for some specific point in history.
This "meta_data" is only available in one specific journal-mode --> "wal"

For any other Journal-modes it's not available IF the transaction has been commited
https://www.sqlite.org/pragma.html#pragma_journal_mode
Quote
The DELETE journaling mode is the normal behavior. In the DELETE mode, the rollback journal is deleted at the conclusion of each transaction. Indeed, the delete operation is the action that causes the transaction to commit.
« Last Edit: February 13, 2024, 12:06:10 pm by Zvoni »
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

 

TinyPortal © 2005-2018