Forum > Database

tbufdataset.CopyfromDataset - strange behaviour in Lazarus trunk version

<< < (2/3) > >>

Thank you for your reply.

Your answer led me to further test using other memorydataset. The results are  as follows:
1. zmQueryDataSet (a descendant of TBufDataset) - same issue
2. rxmemdataset - No issue
3. zMemTable (new component in the zeoslib) - No issue.

Based on the above, I think the problem comes from TBufDataset. Is anyone maintaining this component (to whom we can highlight this issue)  currently?


--- Quote from: kjteng on November 03, 2021, 10:21:51 am ---Is anyone maintaining this component (to whom we can highlight this issue)  currently?

--- End quote ---
Most of the database-related bugs are handled by Michel Van Canneyt. But I am rather sure that he will not feel responsible for the issue when it can only be demonstated on the basis of a ZEOS database...

I tried to further simplify the problem and modified your demo so that it creates a simple ZEOS-DB with a single ftLargeInt column. And this one cannot be copied to a TBufDataset either, the field does appear but is empty; and when the program ends, there is a memory access error. These are indications that the buffers are not set up correctly in the BufDataset.

Two observations:

(1) When I replace the ZQuery by a TMemoryDataset the issue is gone.

(2) My program can use the TBufDataset of FPC 3.2.2 or FPC 3.3.1 (I copied the sources to subdirs of the project). Both versions show the same error with FPC 3.3.1. But, when I compile with FPC 3.2 the error does not appear, neither with the BufDataset of 3.3.1 nor with 3.2.2. This seems to indicate that the issue does not originate in the TBufdataset sources, but somewhere else, probably in unit db or related, or even completely somewhere else in FPC.

I think I found the issue. It's the FieldNo property. It seems to be that ZEOS is counting the FieldNo starting at 0, but LCL documentation says that it is a 1-based property (, and in fact, there are lots of lines in the FPC database code in which Field.FieldNo-1 is calculated.

Please copy the following code into your test program and replace the  bf1.CopyFromDataset(ZReadOnlyQuery1,true) by CopyFromDataset(ZReadOnlyQuery1, bf1, true). This modified code detects whether the FieldNo is 0-based and increments it in this case (the changed lines are highlighted).

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---procedure CopyFromDataset(ASource, ADest: TDataSet; CopyData: Boolean);const  UseStreams = ftBlobTypes;var  I: Integer;  F,F1,F2: TField;  L1,L2: TList;  N: String;  OriginalPosition: TBookMark;  S: TMemoryStream;  minFieldNo: Integer;  fieldNo: Integer;begin  ADest.Close;  ADest.Fields.Clear;  ADest.FieldDefs.Clear;   minFieldNo := MaxInt;  for I:=0 to ASource.FieldCount-1 do  begin    F := ASource.Fields[i];    if F.FieldNo < minFieldNo then      minFieldNo := F.FieldNo;  end;   for I:=0 to ASource.FieldCount-1 do  begin    F := ASource.Fields[I];    fieldNo := F.FieldNo;    if minFieldNo = 0 then inc(fieldno);    TFieldDef.Create(ADest.FieldDefs, F.FieldName, F.DataType, F.Size, F.Required, fieldno);  end;  if (ADest is TBufDataset) then    TBufDataset(ADest).CreateDataset  else    raise Exception.Create('Destination dataset type not supported.');  L1 := nil;  L2 := nil;  S := nil;  if CopyData then    try      L1:=TList.Create;      L2:=TList.Create;      ADest.Open;      for I:=0 to ADest.FieldDefs.Count-1 do      begin        N := ADest.FieldDefs[I].Name;        F1 := ADest.FieldByName(N);        F2 := ASource.FieldByName(N);        L1.Add(F1);        L2.Add(F2);        If (ADest.FieldDefs[I].DataType in UseStreams) and (S=Nil) then          S := TMemoryStream.Create;      end;      OriginalPosition := ASource.GetBookmark;      try        ASource.Open;        ASource.First;        while not ASource.EOF do        begin          ADest.Append;          for I:=0 to L1.Count-1 do          begin            F1 := TField(L1[i]);            F2 := TField(L2[I]);            if not F2.IsNull then              case F1.DataType of                ftFixedChar,                ftString   : F1.AsString:=F2.AsString;                ftFixedWideChar,                ftWideString : F1.AsWideString:=F2.AsWideString;                ftBoolean  : F1.AsBoolean:=F2.AsBoolean;                ftFloat    : F1.AsFloat:=F2.AsFloat;                ftAutoInc,                ftSmallInt,                ftInteger  : F1.AsInteger:=F2.AsInteger;                ftLargeInt : F1.AsLargeInt:=F2.AsLargeInt;                ftDate     : F1.AsDateTime:=F2.AsDateTime;                ftTime     : F1.AsDateTime:=F2.AsDateTime;                ftTimestamp,                ftDateTime : F1.AsDateTime:=F2.AsDateTime;                ftCurrency : F1.AsCurrency:=F2.AsCurrency;                ftBCD,                ftFmtBCD   : F1.AsBCD:=F2.AsBCD;              else                if (F1.DataType in UseStreams) then                begin                  S.Clear;                  TBlobField(F2).SaveToStream(S);                  S.Position := 0;                  TBlobField(F1).LoadFromStream(S);                end                else                  F1.AsString := F2.AsString;              end;          end;          try            ADest.Post;          except            ADest.Cancel;            raise;          end;          ASource.Next;        end;      finally        ASource.GotoBookmark(OriginalPosition); //Return to original record      end;    finally      L2.Free;      l1.Free;      S.Free;    end;end;
After fixing the FieldNo this way the ZEOS file can be copied successfully.

Reading some comments in the ZEOS code I found that they intentionally switch from 1-based to 0-based indexes. I reported this thread to the ZEOS forum as an example that this change still has issues (

wp, Thank you for the fix. Now it copydata correctly. I have removed the line 'if CopyData then...'
because it wont compile with this line. What would be the effect without this line?


[0] Message Index

[#] Next page

[*] Previous page

Go to full version