Lazarus

Programming => General => Topic started by: mangakissa on October 23, 2020, 01:28:38 pm

Title: TStringlist is not split correctly
Post by: mangakissa on October 23, 2020, 01:28:38 pm
As I told earlier, I found a bug in TStringlist. But this is general in pascal, not only Lazarus / FPC.
I have this piece of code:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. var slTest  : TStringlist;
  6.     slRegel : TStringlist;
  7.     index   : integer;
  8. begin
  9.   slTest  := TStringlist.create;
  10.   slRegel := TStringlist.create;
  11.   try
  12.     slTest.LoadFromFile('test.txt');
  13.     slRegel.Delimiter := ';';
  14.     slRegel.QuoteChar := '"';
  15.     slRegel.StrictDelimiter := true;
  16.     for index := 0 to slTest.count - 1 do
  17.     begin
  18.       slRegel.DelimitedText := slTest[index]);
  19.       writeln(inttostr(slRegel.Count))
  20.     end;
  21.     readln();
  22.   finally
  23.     slregel.free;
  24.     sltest.free;
  25.   end;
  26.  
  27. end.                                          
  28.  
This is my textfile:
Code: Text  [Select][+][-]
  1. field11;field12        ;field13;field14
  2. field21;field22        ;field23;field24
  3. field31;"field32"      ;field33;field34
  4. field41;field42        ;field43;field44
  5.  
When I run this sample, the following results in:
4
4
5
4

The reason is the double quotes together with a few spaces afterwards. TStringlist creates an extra entry.

Am I right or not?

The workaround is:
Code: Pascal  [Select][+][-]
  1. slRegel.DelimitedText := StringReplace(slTest[index],'"','',[rfReplaceAll]);
  2.  

But using strictdelimiter should it work with this workaround.
Title: Re: TStringlist is not split correctly
Post by: wp on October 23, 2020, 05:20:50 pm
This seems to be a bug. Please report it and provide an example (like yours).

As a workaround you can use the .Split string help which seems to behave correctly:

Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. const
  6.   text: String =
  7.     'field11;field12        ;field13;field14' + LineEnding +
  8.     'field21;field22        ;field23;field24' + LineEnding +
  9.     'field31;"field32"      ;field33;field34' + LineEnding +
  10.     'field41;field42        ;field43;field44';
  11.  
  12. var
  13.   slTest: TStringList;
  14.   stream: TStringStream;
  15.   sa: TStringArray;
  16.   i: Integer;
  17.  
  18. begin
  19.   slTest := TStringList.Create;
  20.   try
  21.     stream := TStringStream.Create(text);
  22.     try
  23.       slTest.LoadfromStream(stream);
  24.     finally
  25.       stream.Free;
  26.     end;
  27.  
  28.     for i := 0 to slTest.Count-1 do
  29.     begin
  30.       sa := slTest[i].Split(';', '"');
  31.       WriteLn(Length(sa));
  32.     end;
  33.   finally
  34.     slTest.Free;
  35.   end;
  36.  
  37.   ReadLn;
  38. end.              
Title: Re: TStringlist is not split correctly
Post by: jamie on October 23, 2020, 11:34:26 pm
Quote from: mangakissa on October 23, 2020, 01:28:38 pm
As I told earlier, I found a bug in TStringlist. But this is general in pascal, not only Lazarus / FPC.
I have this piece of code:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. var slTest  : TStringlist;
  6.     slRegel : TStringlist;
  7.     index   : integer;
  8. begin
  9.   slTest  := TStringlist.create;
  10.   slRegel := TStringlist.create;
  11.   try
  12.     slTest.LoadFromFile('test.txt');
  13.     slRegel.Delimiter := ';';
  14.     slRegel.QuoteChar := '"';
  15.     slRegel.StrictDelimiter := true;
  16.     for index := 0 to slTest.count - 1 do
  17.     begin
  18.       slRegel.DelimitedText := slTest[index]);
  19.       writeln(inttostr(slRegel.Count))
  20.     end;
  21.     readln();
  22.   finally
  23.     slregel.free;
  24.     sltest.free;
  25.   end;
  26.  
  27. end.                                          
  28.  
This is my textfile:
Code: Text  [Select][+][-]
  1. field11;field12        ;field13;field14
  2. field21;field22        ;field23;field24
  3. field31;"field32"      ;field33;field34
  4. field41;field42        ;field43;field44
  5.  
When I run this sample, the following results in:
4
4
5
4

The reason is the double quotes together with a few spaces afterwards. TStringlist creates an extra entry.

Am I right or not?

The workaround is:
Code: Pascal  [Select][+][-]
  1. slRegel.DelimitedText := StringReplace(slTest[index],'"','',[rfReplaceAll]);
  2.  

But using strictdelimiter should it work with this workaround.

It's a known problem with the stinglist, at least it's very well known to me..
it seems to work in Delphi but not in fpc when encountering spaces like that and getting an extra line inserted.

Which is why I wrote my own TlinesString class based from Tstrings. It basically works the same but it actually functions correctly.

Its does not handle strings internally like a Stringlist does but instead builds a text table as a single chunk of memory, unlike what stringlist does.

 But you still need stringlist if you want to store arbitrary data in it where as what I did values like #0 are not allowed in strings as a value.

I voiced my opinion already about this and all I got was rocks thrown back at me. so here we are..

I suggest maybe writing your own parser of SPLIT does not work correctly for you in the helpers.

Title: Re: TStringlist is not split correctly
Post by: winni on October 24, 2020, 12:09:44 am
Hi!

Changed your first code to a procedure and loaded the const text
The result is 4 / 4 / 4 / 4

Changed  your first code to rrunning program. Without compiler errors.
Threw out the stream and the try.
Result is 4 / 4 / 4 / 4

fpc 3.04, lin 64

Winni
Title: Re: TStringlist is not split correctly
Post by: wp on October 24, 2020, 11:45:54 am
Quote from: winni on October 24, 2020, 12:09:44 am
Changed your first code to a procedure and loaded the const text
The result is 4 / 4 / 4 / 4

Changed  your first code to rrunning program. Without compiler errors.
Threw out the stream and the try.
Result is 4 / 4 / 4 / 4

fpc 3.04, lin 64

I don't know what you are doing exactly. Here is what I am doing (just paste it into the editor window of a new Lazarus "Simple Program" project), and the output is 4 / 4 / 5 / 4. I am on Windows 10 (64 bit), Laz trunk, fpc 3.2.0 (32 bit or 64 bit), or fpc 3.0.4 (32 bit or 64 bit), no difference.
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. const
  6.   TEXT =
  7.       'field11;field12        ;field13;field14' + LineEnding +
  8.       'field21;field22        ;field23;field24' + LineEnding +
  9.       'field31;"field32"      ;field33;field34' + LineEnding +
  10.       'field41;field42        ;field43;field44';
  11.  
  12. var slTest  : TStringlist;
  13.     slRegel : TStringlist;
  14.     index   : integer;
  15. begin
  16.   slTest  := TStringlist.create;
  17.   slRegel := TStringlist.create;
  18.   try
  19. //    slTest.LoadFromFile('test.txt');
  20.     slTest.Text := TEXT;
  21.     slRegel.Delimiter := ';';
  22.     slRegel.QuoteChar := '"';
  23.     slRegel.StrictDelimiter := true;
  24.     for index := 0 to slTest.count - 1 do
  25.     begin
  26.       slRegel.DelimitedText := slTest[index];
  27.       writeln(inttostr(slRegel.Count))
  28.     end;
  29.     readln();
  30.   finally
  31.     slregel.free;
  32.     sltest.free;
  33.   end;
  34. end.

I submitted a bug report with a simple test program (https://bugs.freepascal.org/view.php?id=37978).
Title: Re: TStringlist is not split correctly
Post by: jamie on October 24, 2020, 09:10:04 pm
attached is a unit file "Linestrings"

It works basically the same as TStringList and is assignable to memos and other tString classes

but the point is, you can try it for your problem or even look at the code that handles it..

I just compared this to Delphi and it seems to be very close for the time being. I am sure there are things missing, there usually are..
Title: Re: TStringlist is not split correctly
Post by: mangakissa on October 25, 2020, 09:04:17 am
Quote
I submitted a bug report with a simple test program (https://bugs.freepascal.org/view.php?id=37978).
As I thought this problem has a really minor chance to fix. It's a general problem. Thanks for trying, WP.
TinyPortal © 2005-2018