Recent

Author Topic: TStringlist is not split correctly  (Read 507 times)

mangakissa

  • Hero Member
  • *****
  • Posts: 1101
TStringlist is not split correctly
« on: October 23, 2020, 01:28:38 pm »
As I told earlier, I found a bug in TStringlist. But this is general in pascal, not only Lazarus / FPC.
I have this piece of code:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. var slTest  : TStringlist;
  6.     slRegel : TStringlist;
  7.     index   : integer;
  8. begin
  9.   slTest  := TStringlist.create;
  10.   slRegel := TStringlist.create;
  11.   try
  12.     slTest.LoadFromFile('test.txt');
  13.     slRegel.Delimiter := ';';
  14.     slRegel.QuoteChar := '"';
  15.     slRegel.StrictDelimiter := true;
  16.     for index := 0 to slTest.count - 1 do
  17.     begin
  18.       slRegel.DelimitedText := slTest[index]);
  19.       writeln(inttostr(slRegel.Count))
  20.     end;
  21.     readln();
  22.   finally
  23.     slregel.free;
  24.     sltest.free;
  25.   end;
  26.  
  27. end.                                          
  28.  
This is my textfile:
Code: Text  [Select][+][-]
  1. field11;field12        ;field13;field14
  2. field21;field22        ;field23;field24
  3. field31;"field32"      ;field33;field34
  4. field41;field42        ;field43;field44
  5.  
When I run this sample, the following results in:
4
4
5
4

The reason is the double quotes together with a few spaces afterwards. TStringlist creates an extra entry.

Am I right or not?

The workaround is:
Code: Pascal  [Select][+][-]
  1. slRegel.DelimitedText := StringReplace(slTest[index],'"','',[rfReplaceAll]);
  2.  

But using strictdelimiter should it work with this workaround.
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

wp

  • Hero Member
  • *****
  • Posts: 7750
Re: TStringlist is not split correctly
« Reply #1 on: October 23, 2020, 05:20:50 pm »
This seems to be a bug. Please report it and provide an example (like yours).

As a workaround you can use the .Split string help which seems to behave correctly:

Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. const
  6.   text: String =
  7.     'field11;field12        ;field13;field14' + LineEnding +
  8.     'field21;field22        ;field23;field24' + LineEnding +
  9.     'field31;"field32"      ;field33;field34' + LineEnding +
  10.     'field41;field42        ;field43;field44';
  11.  
  12. var
  13.   slTest: TStringList;
  14.   stream: TStringStream;
  15.   sa: TStringArray;
  16.   i: Integer;
  17.  
  18. begin
  19.   slTest := TStringList.Create;
  20.   try
  21.     stream := TStringStream.Create(text);
  22.     try
  23.       slTest.LoadfromStream(stream);
  24.     finally
  25.       stream.Free;
  26.     end;
  27.  
  28.     for i := 0 to slTest.Count-1 do
  29.     begin
  30.       sa := slTest[i].Split(';', '"');
  31.       WriteLn(Length(sa));
  32.     end;
  33.   finally
  34.     slTest.Free;
  35.   end;
  36.  
  37.   ReadLn;
  38. end.              
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

jamie

  • Hero Member
  • *****
  • Posts: 3797
Re: TStringlist is not split correctly
« Reply #2 on: October 23, 2020, 11:34:26 pm »
As I told earlier, I found a bug in TStringlist. But this is general in pascal, not only Lazarus / FPC.
I have this piece of code:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. var slTest  : TStringlist;
  6.     slRegel : TStringlist;
  7.     index   : integer;
  8. begin
  9.   slTest  := TStringlist.create;
  10.   slRegel := TStringlist.create;
  11.   try
  12.     slTest.LoadFromFile('test.txt');
  13.     slRegel.Delimiter := ';';
  14.     slRegel.QuoteChar := '"';
  15.     slRegel.StrictDelimiter := true;
  16.     for index := 0 to slTest.count - 1 do
  17.     begin
  18.       slRegel.DelimitedText := slTest[index]);
  19.       writeln(inttostr(slRegel.Count))
  20.     end;
  21.     readln();
  22.   finally
  23.     slregel.free;
  24.     sltest.free;
  25.   end;
  26.  
  27. end.                                          
  28.  
This is my textfile:
Code: Text  [Select][+][-]
  1. field11;field12        ;field13;field14
  2. field21;field22        ;field23;field24
  3. field31;"field32"      ;field33;field34
  4. field41;field42        ;field43;field44
  5.  
When I run this sample, the following results in:
4
4
5
4

The reason is the double quotes together with a few spaces afterwards. TStringlist creates an extra entry.

Am I right or not?

The workaround is:
Code: Pascal  [Select][+][-]
  1. slRegel.DelimitedText := StringReplace(slTest[index],'"','',[rfReplaceAll]);
  2.  

But using strictdelimiter should it work with this workaround.

It's a known problem with the stinglist, at least it's very well known to me..
it seems to work in Delphi but not in fpc when encountering spaces like that and getting an extra line inserted.

Which is why I wrote my own TlinesString class based from Tstrings. It basically works the same but it actually functions correctly.

Its does not handle strings internally like a Stringlist does but instead builds a text table as a single chunk of memory, unlike what stringlist does.

 But you still need stringlist if you want to store arbitrary data in it where as what I did values like #0 are not allowed in strings as a value.

I voiced my opinion already about this and all I got was rocks thrown back at me. so here we are..

I suggest maybe writing your own parser of SPLIT does not work correctly for you in the helpers.

The only true wisdom is knowing you know nothing

winni

  • Hero Member
  • *****
  • Posts: 2009
Re: TStringlist is not split correctly
« Reply #3 on: October 24, 2020, 12:09:44 am »
Hi!

Changed your first code to a procedure and loaded the const text
The result is 4 / 4 / 4 / 4

Changed  your first code to rrunning program. Without compiler errors.
Threw out the stream and the try.
Result is 4 / 4 / 4 / 4

fpc 3.04, lin 64

Winni

wp

  • Hero Member
  • *****
  • Posts: 7750
Re: TStringlist is not split correctly
« Reply #4 on: October 24, 2020, 11:45:54 am »
Changed your first code to a procedure and loaded the const text
The result is 4 / 4 / 4 / 4

Changed  your first code to rrunning program. Without compiler errors.
Threw out the stream and the try.
Result is 4 / 4 / 4 / 4

fpc 3.04, lin 64

I don't know what you are doing exactly. Here is what I am doing (just paste it into the editor window of a new Lazarus "Simple Program" project), and the output is 4 / 4 / 5 / 4. I am on Windows 10 (64 bit), Laz trunk, fpc 3.2.0 (32 bit or 64 bit), or fpc 3.0.4 (32 bit or 64 bit), no difference.
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. uses classes, sysutils;
  4.  
  5. const
  6.   TEXT =
  7.       'field11;field12        ;field13;field14' + LineEnding +
  8.       'field21;field22        ;field23;field24' + LineEnding +
  9.       'field31;"field32"      ;field33;field34' + LineEnding +
  10.       'field41;field42        ;field43;field44';
  11.  
  12. var slTest  : TStringlist;
  13.     slRegel : TStringlist;
  14.     index   : integer;
  15. begin
  16.   slTest  := TStringlist.create;
  17.   slRegel := TStringlist.create;
  18.   try
  19. //    slTest.LoadFromFile('test.txt');
  20.     slTest.Text := TEXT;
  21.     slRegel.Delimiter := ';';
  22.     slRegel.QuoteChar := '"';
  23.     slRegel.StrictDelimiter := true;
  24.     for index := 0 to slTest.count - 1 do
  25.     begin
  26.       slRegel.DelimitedText := slTest[index];
  27.       writeln(inttostr(slRegel.Count))
  28.     end;
  29.     readln();
  30.   finally
  31.     slregel.free;
  32.     sltest.free;
  33.   end;
  34. end.

I submitted a bug report with a simple test program (https://bugs.freepascal.org/view.php?id=37978).
« Last Edit: October 24, 2020, 12:04:06 pm by wp »
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

jamie

  • Hero Member
  • *****
  • Posts: 3797
Re: TStringlist is not split correctly
« Reply #5 on: October 24, 2020, 09:10:04 pm »
attached is a unit file "Linestrings"

It works basically the same as TStringList and is assignable to memos and other tString classes

but the point is, you can try it for your problem or even look at the code that handles it..

I just compared this to Delphi and it seems to be very close for the time being. I am sure there are things missing, there usually are..
The only true wisdom is knowing you know nothing

mangakissa

  • Hero Member
  • *****
  • Posts: 1101
Re: TStringlist is not split correctly
« Reply #6 on: October 25, 2020, 09:04:17 am »
Quote
I submitted a bug report with a simple test program (https://bugs.freepascal.org/view.php?id=37978).
As I thought this problem has a really minor chance to fix. It's a general problem. Thanks for trying, WP.
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

 

TinyPortal © 2005-2018