Problem with TFixedFormatDataSet

[janhsh]

In a form, I place a TFixedFormatDataSet
with schema:

P=18
E=17
N=24
Z=19
D=10

which corresponds to the following fixed column data file

Code: Text  [Select][+][-]

1. 1                 21               31                      41                 51        
2. 2                 22               32                      42                 52        
3. 3                 23               33                      43                 53        
4. 4                 24               34                      44                 54        
5. 5                 25               35                      45                 55        
6. 6                 26               36                      46                 56        
7. 7                 27               37                      47                 57        
8. 8                 28               38                      48                 58        
9. 9                 29               39                      49                 59        
10. 10                210              310                     410                510      
11. 11                211              311                     411                511    

[/font] 

After activation, FieldDefs has 5 items that have a correct size.

So I should have a correct split of the columns of the file but instead, I have the result displayed in the image


Has anyone ever used a TFixedFormatDataSet to read data from a fixed column file?

[paweld]

Set theCodePage property to 'windows-1252' and then it should work

[janhsh]

It works for numeric values, but my file is originally encoded in utf-8.

So if I put text values ​​in my file, the splitting remains correct but the characters are misinterpreted

[CharlyTango]

I think this has something to do with the encoding UTF8 not all letters have the length 1.

I would use a delimited format instead of the fixed format dataset and would work with TBufDataset or at least TMemDataset.

[Richard Marriott]

I created the file through the IDE and everything works as expected.
As others have suggested the format of your text file is possibly not ASCII/ANSI. Try opening it in a text editor (say NotePad) to see if it is actually in fixed width format (tabular). If so TFixedFormatDataset should be able to read it. Here is a link to a website/blog that has examples of how to get all of the TDataset descendants that ship with Lazarus to work (in code and through the IDE). Maybe you can spot what the issue is there. Good luck.
https://tdataset7.wordpress.com/

[wp]

I wrote a test project for well-defined data files, one encoded in code page 1252 and one in utf-8. The fields are determined correctly for cp1252, but are too wide by a factor 4 in case of the utf8 file.

The FixedFormatDataset seems to incorrectly extract the fields from utf-8 encoded files. A specialty of utf-8 files is that the fields in the record buffer are dimensioned for four times their field size. This is a safety measure to guarantee that all utf-8 codepoints are stored correctly in the buffer. This factor is not correctly considered when the field values are extracted.

Filed a bug report (https://gitlab.com/freepascal.org/fpc/source/-/issues/40911)

[wp]

Filed a bug report (https://gitlab.com/freepascal.org/fpc/source/-/issues/40911)

Michael Van Canneyt responded: "UTF8 can never be fixed format, so I do not intend to fix this."