Lazarus

Programming => Databases => Topic started by: dmitryb on January 24, 2023, 07:18:16 pm

Title: TBufDataset.First issue.
Post by: dmitryb on January 24, 2023, 07:18:16 pm
Lazarus 2.2.4 (rev lazarus_2_2_4) FPC 3.2.2 x86_64-win64-win32/win64

Next code in Lazarus 2.2.4  shows empty string.

Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button4Click(Sender: TObject);
  2. var
  3.   s: String;
  4. begin
  5.   BufDataset1 := TBufDataset.Create(Self);
  6.  
  7.   BufDataset1.FieldDefs.Add('Str5Field', ftString, 5, -1, False, False, 0, CP_UTF8);
  8.   BufDataset1.CreateDataset;
  9.  
  10.   BufDataset1.Append;
  11.   BufDataset1.FieldByName('Str5Field').AsString := 'ABCDEFG';
  12.   BufDataset1.Post;
  13.   BufDataset1.First;
  14.  
  15.   s := VarToStr(BufDataset1.FieldByName('Str5Field').AsVariant);
  16.  
  17.   ShowMessage(s);
  18. end;
  19.  

Is this a feature or a bug?
Title: Re: TBufDataset.First issue.
Post by: egsuh on January 25, 2023, 04:26:46 am
Why not simply

     s := BufDataset1.FieldByName('Str5Field').AsString;

?
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 09:21:08 am
Quote from: egsuh on January 25, 2023, 04:26:46 am
Why not simply

     s := BufDataset1.FieldByName('Str5Field').AsString;

?
Same issue.

I can confirm
Code: Pascal  [Select][+][-]
  1. program Project1;
  2. Uses bufdataset,db;
  3.  
  4. Var
  5.   s:String;
  6.   bd:TBufDataset;
  7.  
  8. begin
  9.   bd:=TBufDataset.Create(Nil);
  10.   bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,0,CP_UTF8);
  11.   bd.CreateDataset;
  12.   Writeln('RecCount=',bd.RecordCount);
  13.   bd.Append;
  14.   bd.FieldByName('Str5Field').AsString:='ABCDEFG';
  15.   bd.Post;
  16.   Writeln('RecCount=',bd.RecordCount);
  17.   Writeln('BOF=',bd.BOF,' - EOF=',bd.EOF);
  18.   s:=bd.Fieldbyname('Str5Field').AsString;
  19.   Writeln('recNo=',bd.RecNo,' - s=',s);
  20.   bd.First;
  21.   Writeln('BOF=',bd.BOF,' - EOF=',bd.EOF);
  22.   s:=bd.Fieldbyname('Str5Field').AsString;
  23.   Writeln('recNo=',bd.RecNo,' - s=',s);
  24.   bd.Last;
  25.   Writeln('BOF=',bd.BOF,' - EOF=',bd.EOF);
  26.   s:=bd.Fieldbyname('Str5Field').AsString;
  27.   Writeln('recNo=',bd.RecNo,' - s=',s);
  28.   bd.First;
  29.   Writeln('BOF=',bd.BOF,' - EOF=',bd.EOF);
  30.   s:=bd.Fieldbyname('Str5Field').AsString;
  31.   Writeln('recNo=',bd.RecNo,' - s=',s);
  32. end.

Result
Quote
RecCount=0
RecCount=1
BOF=FALSE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=
BOF=FALSE - EOF=TRUE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=
Title: Re: TBufDataset.First issue.
Post by: TRon on January 25, 2023, 09:46:15 am
In addition to Zvoni's findings:
- it leaks memory
- it creates an access violation when not using a exception block to free the instantiated class.

I get very strange output characters instead of receiving empty strings (unlike Zvoni)
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 10:14:43 am
Quote from: TRon on January 25, 2023, 09:46:15 am
In addition to Zvoni's findings:
- it leaks memory
- it creates an access violation when not using a exception block to free the instantiated class.
Yes, it's the reason why i left out bd.free in above code (and i don't care about memory-leaks when just testing around and it's basically the end of the program anyway)

Something that baffles me: RecCount is 1, shouldn't BOF and EOF both be true, irrespective if i move Next, Prior, First or Last??
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 10:22:21 am
The problem might be that you are trying to stuff a string of 7 characters "ABCDEFG" in a field defined with Size = 5 !! %)

That really messes things up.

Just define the stringfield as a field of 20 characters and the code should work fine.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,0,CP_UTF8);
  2.   bd.FieldDefs.Add('Str5Field', ftString, 20);
  3.  
(I'm also not sure about the other parameters you used in the field-def. But the simple version I used above works.)
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 10:24:08 am
Quote from: rvk on January 25, 2023, 10:22:21 am
The problem might be that you are trying to stuff a string of 7 characters "ABCDEFG" in a field defined with Size = 5 !! %)

That really messes things up.

Just define the stringfield as a field of 20 characters and the code should work fine.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,0,CP_UTF8);
  2.   bd.FieldDefs.Add('Str5Field', ftString, 20);
  3.  
Was my first thought, too, but doesn't make a difference if you shorten the String to "ABCDE" - same issue.
Just try it.

EDIT: What i noticed: The issue appears when BOF=True, irrespective of the Value of EOF
It returns the correct Value for BOF=False
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 10:26:08 am
Quote from: Zvoni on January 25, 2023, 10:24:08 am
Quote from: rvk on January 25, 2023, 10:22:21 am
The problem might be that you are trying to stuff a string of 7 characters "ABCDEFG" in a field defined with Size = 5 !! %)

That really messes things up.

Just define the stringfield as a field of 20 characters and the code should work fine.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,0,CP_UTF8);
  2.   bd.FieldDefs.Add('Str5Field', ftString, 20);
  3.  
Was my first thought, too, but doesn't make a difference if you shorten the String to "ABCDE" - same issue.
Just try it.
And what happens if you use this??
Code: Pascal  [Select][+][-]
  1.  bd.FieldDefs.Add('Str5Field', ftString, 20);

The problem could also be in the precision of -1 or something else.
Using the simple Add() version works for me.

O, or FieldNo. Why is 0 used here?
Shouldn't this be 1????
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 10:28:27 am
Quote from: rvk on January 25, 2023, 10:26:08 am
Quote from: Zvoni on January 25, 2023, 10:24:08 am
Quote from: rvk on January 25, 2023, 10:22:21 am
The problem might be that you are trying to stuff a string of 7 characters "ABCDEFG" in a field defined with Size = 5 !! %)

That really messes things up.

Just define the stringfield as a field of 20 characters and the code should work fine.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,0,CP_UTF8);
  2.   bd.FieldDefs.Add('Str5Field', ftString, 20);
  3.  
Was my first thought, too, but doesn't make a difference if you shorten the String to "ABCDE" - same issue.
Just try it.
And what happens if you use this??
Code: Pascal  [Select][+][-]
  1.  bd.FieldDefs.Add('Str5Field', ftString, 20);

The problem could also be in the precision of -1 or something else.
Using the simple Add() version works for me.
Yes, i wondered about that, too (since when do you need precision for Strings?!?!), but i think OP used this overloaded function because of the following params after precision (required, readonly, codepage)
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 10:29:57 am
Quote from: Zvoni on January 25, 2023, 10:28:27 am
Yes, i wondered about that, too (since when do you need precision for Strings?!?!), but i think OP used this overloaded function because of the following params after precision (required, readonly, codepage)
I think the fieldNo of 0 is the problem.
Make it one (1) and it works.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field', ftString, 20);
  2.   bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,1,CP_UTF8);
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 10:34:34 am
Quote from: rvk on January 25, 2023, 10:29:57 am
Quote from: Zvoni on January 25, 2023, 10:28:27 am
Yes, i wondered about that, too (since when do you need precision for Strings?!?!), but i think OP used this overloaded function because of the following params after precision (required, readonly, codepage)
I think the fieldNo of 0 is the problem.
Make it one (1) and it works.

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field', ftString, 20);
  2.   bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,1,CP_UTF8);

Yep. From the source:
Quote
  // the fielddef will register itself here as an owned component.
  // fieldno is 1 based !
So don't use field 0 but start on 1 !!!!

Not sure if there is any documentation for this  :P
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 10:40:48 am
Quote
Quote from: rvk on January 25, 2023, 10:34:34 am

Code: Pascal  [Select][+][-]
  1.   // bd.FieldDefs.Add('Str5Field', ftString, 20);
  2.   bd.FieldDefs.Add('Str5Field',ftString,5,-1,False,False,1,CP_UTF8);

Yep. From the source:
Quote
  // the fielddef will register itself here as an owned component.
  // fieldno is 1 based !
So don't use field 0 but start on 1 !!!!

Not sure if there is any documentation for this  :P
Yep, that's it.

Result
Quote
RecCount=0
RecCount=1
BOF=FALSE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=FALSE - EOF=TRUE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
Still doesn't explain the BOF/EOF confusion
Title: Re: TBufDataset.First issue.
Post by: TRon on January 25, 2023, 10:45:05 am
Confirmed that rvk's solution works.

I agree with Zvoni wrt documentation (I was unable to locate it, though the property is named fielNo as in number not index). Even a exception on providing a 'illegal' field number would have been helpful.

I seem to remember there being something being off with eof/bof. From what I remember it was logical it acted this way (I seem unable to locate the conversation as I was not part of it, should be here on the forum)
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 10:45:44 am
Quote from: Zvoni on January 25, 2023, 10:40:48 am
Result
Quote
RecCount=0
RecCount=1
BOF=FALSE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=FALSE - EOF=TRUE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
Still doesn't explain the BOF/EOF confusion
What confusion?
They are correct in this example. (or am I missing something?)

BOF and EOF are not set until they are actually encountered.
With a POST they are not set. Only with Last, First, Next and Prior (and only in the direction you where going).
Title: Re: TBufDataset.First issue.
Post by: wp on January 25, 2023, 10:45:58 am
Quote from: rvk on January 25, 2023, 10:34:34 am
Not sure if there is any documentation for this  :P
https://www.freepascal.org/docs-html/fcl/db/tfield.fieldno.html ("It is a 1-based index")
Title: Re: TBufDataset.First issue.
Post by: TRon on January 25, 2023, 10:48:23 am
Quote from: wp on January 25, 2023, 10:45:58 am
Quote from: rvk on January 25, 2023, 10:34:34 am
Not sure if there is any documentation for this  :P
https://www.freepascal.org/docs-html/fcl/db/tfield.fieldno.html ("It is a 1-based index")
Ok, I came in from another end: https://www.freepascal.org/docs-html/fcl/db/tfielddef.fieldno.html
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 11:04:36 am
Quote from: rvk on January 25, 2023, 10:45:44 am
Quote from: Zvoni on January 25, 2023, 10:40:48 am
Result
Quote
RecCount=0
RecCount=1
BOF=FALSE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
BOF=FALSE - EOF=TRUE
recNo=1 - s=ABCDEFG
BOF=TRUE - EOF=FALSE
recNo=1 - s=ABCDEFG
Still doesn't explain the BOF/EOF confusion
What confusion?
They are correct in this example. (or am I missing something?)

BOF and EOF are not set until they are actually encountered.
With a POST they are not set. Only with Last, First, Next and Prior (and only in the direction you where going).
For a RecordCount = 1 i would expect BOF and EOF to be True.
Or at least EOF to be True from the get-go, since OP used Append
Title: Re: TBufDataset.First issue.
Post by: rvk on January 25, 2023, 11:11:07 am
Quote from: Zvoni on January 25, 2023, 11:04:36 am
For a RecordCount = 1 i would expect BOF and EOF to be True.
No. BOF and EOF are only set on movement of the cursor.
Quote
1. The cursor is on the last record, and the TDataset.Next method is called.
https://www.freepascal.org/docs-html/fcl/db/tdataset.eof.html

So, although you are on the last record after Append with one record, Next isn't called, so EOF is false.

Although the documentation page of BOF (https://www.freepascal.org/docs-html/fcl/db/tdataset.bof.html) isn't that clear, the same as for EOF goes for BOF.

Only exception is when there is an empty dataset. Then both BOF and EOF are true.

Also note:
Quote
In all other cases, EOF is False. Note: when the cursor is on the last-but-one record, and Next is called (moving the cursor to the last record), EOF will not yet be True.
So, EOF will only trigger if you are trying to go past the last record. Not when you reach that last record.
(That's why you do a while EOF loop with Next on the end, otherwise this would go wrong)
Title: Re: TBufDataset.First issue.
Post by: wp on January 25, 2023, 11:15:42 am
Quote from: TRon on January 25, 2023, 10:48:23 am
Ok, I came in from another end: https://www.freepascal.org/docs-html/fcl/db/tfielddef.fieldno.html
I agree that it is "poorly documented". I just know this because I once was involved in a bug fix related to ZEOS components which changed their RecNo to begin at 0, and this caused some trouble (https://sourceforge.net/p/zeoslib/tickets/539/).
Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 11:32:31 am
Quote from: rvk on January 25, 2023, 11:11:07 am
Quote from: Zvoni on January 25, 2023, 11:04:36 am
For a RecordCount = 1 i would expect BOF and EOF to be True.
No. BOF and EOF are only set on movement of the cursor.
Quote
1. The cursor is on the last record, and the TDataset.Next method is called.
https://www.freepascal.org/docs-html/fcl/db/tdataset.eof.html

So, although you are on the last record after Append with one record, Next isn't called, so EOF is false.

Although the documentation page of BOF (https://www.freepascal.org/docs-html/fcl/db/tdataset.bof.html) isn't that clear, the same as for EOF goes for BOF.

Only exception is when there is an empty dataset. Then both BOF and EOF are true.

Also note:
Quote
In all other cases, EOF is False. Note: when the cursor is on the last-but-one record, and Next is called (moving the cursor to the last record), EOF will not yet be True.
So, EOF will only trigger if you are trying to go past the last record. Not when you reach that last record.
(That's why you do a while EOF loop with Next on the end, otherwise this would go wrong)

Ah... OK.
I've just read something on Microsofts website, and it is consistent
BOF/EOF will only be set to True if you try to MOVE BEYOND the First/Last record
Title: Re: TBufDataset.First issue.
Post by: TRon on January 25, 2023, 11:53:54 am
Quote from: Zvoni on January 25, 2023, 11:32:31 am
BOF/EOF will only be set to True if you try to MOVE BEYOND the First/Last record
Ah, yes !!. That was the conclusion from the conversation I remembered as well. Thx for the refresh guys/galls !
Title: Re: TBufDataset.First issue.
Post by: Thaddy on January 25, 2023, 02:08:55 pm
An old example from 2016 written by me and already on this forum:
Code: Pascal  [Select][+][-]
  1. {$apptype console}{$mode objfpc}
  2. uses
  3.   db,BufDataset;
  4. var
  5.   BufDb:TBufDataset;
  6. begin
  7.  BufDb:=TBufDataset.Create(nil);
  8.  try
  9.    BufDb.FieldDefs.Add('NAME',ftString,20);
  10.    BufDb.FieldDefs.Add('NUM',ftinteger);
  11.    BufDb.FieldDefs.Add('NUM2',ftInteger);
  12.    BufDb.CreateDataSet;
  13.    BufDb.Open;
  14.    BufDb.Append;
  15.    BufDb.FieldByName('NAME').Value:='Free';
  16.    BufDb.Post;
  17.    BufDb.Append;
  18.    BufDb.FieldByName('NAME').Value:='Pascal';
  19.    BufDb.Post;
  20.    BufDb.SaveToFile('BufDb.txt');
  21.  finally
  22.    BufDb.Close;
  23.    BufDb.Free;
  24.  end;
  25.  // now, open..
  26.  BufDb := TBufDataset.Create(nil);
  27.  try
  28.    BufDb.LoadFromFile('BufDb.txt');
  29.    BufDb.Open;
  30.    BufDb.First;
  31.    Writeln(BufDb.FieldByName('NAME').Value);
  32.  finally
  33.    BufDb.Close;
  34.    BufDb.Free;
  35.  end;
  36.  
  37. end.

It seems to me that just the call to open is the culprit.

https://forum.lazarus.freepascal.org/index.php/topic,33508.msg217317.html#msg217317


Title: Re: TBufDataset.First issue.
Post by: Zvoni on January 25, 2023, 02:30:46 pm
Quote from: Thaddy on January 25, 2023, 02:08:55 pm
An old example from 2016 written by me and already on this forum:
Code: Pascal  [Select][+][-]
  1. {$apptype console}{$mode objfpc}
  2. uses
  3.   db,BufDataset;
  4. var
  5.   BufDb:TBufDataset;
  6. begin
  7.  BufDb:=TBufDataset.Create(nil);
  8.  try
  9.    BufDb.FieldDefs.Add('NAME',ftString,20);
  10.    BufDb.FieldDefs.Add('NUM',ftinteger);
  11.    BufDb.FieldDefs.Add('NUM2',ftInteger);
  12.    BufDb.CreateDataSet;
  13.    BufDb.Open;
  14.    BufDb.Append;
  15.    BufDb.FieldByName('NAME').Value:='Free';
  16.    BufDb.Post;
  17.    BufDb.Append;
  18.    BufDb.FieldByName('NAME').Value:='Pascal';
  19.    BufDb.Post;
  20.    BufDb.SaveToFile('BufDb.txt');
  21.  finally
  22.    BufDb.Close;
  23.    BufDb.Free;
  24.  end;
  25.  // now, open..
  26.  BufDb := TBufDataset.Create(nil);
  27.  try
  28.    BufDb.LoadFromFile('BufDb.txt');
  29.    BufDb.Open;
  30.    BufDb.First;
  31.    Writeln(BufDb.FieldByName('NAME').Value);
  32.  finally
  33.    BufDb.Close;
  34.    BufDb.Free;
  35.  end;
  36.  
  37. end.

It seems to me that just the call to open is the culprit.

https://forum.lazarus.freepascal.org/index.php/topic,33508.msg217317.html#msg217317
No, it was the FieldNo in the (overloaded) Add-Function. OP (and myself) missed that FieldNo is 1-based. OP used for that sample FieldNo=0
Title: Re: TBufDataset.First issue.
Post by: Thaddy on January 25, 2023, 08:00:35 pm
I wouldn't call it no!, because if you want to read the first entry, first will do nicely. Hence that old example.
It is cleaner and much easier than the bull dung above.
Title: Re: TBufDataset.First issue.
Post by: egsuh on January 26, 2023, 03:09:37 am
In Zvony’s example, calling RecNo might have moved cursor beyond last record.
TinyPortal © 2005-2018