Recent

Author Topic: Eliminating empty strings in TStringList when reading from text file.  (Read 21303 times)

MaxCuriosus

  • Full Member
  • ***
  • Posts: 136
In order to eliminate empty lines read from a text file, is there a way to do it without using an explicit loop as in the code below?

Code: Pascal  [Select][+][-]
  1. Var
  2.   SL: TStringList;
  3.   I: Integer;
  4.  
  5. Begin
  6.   SL:=TStringList.Create;
  7.   SL.LoadFromFile(SomeTextFile);
  8.   I:=0;
  9.   While I<SL.Count do
  10.     Begin
  11.       If SL.Strings[I]=''
  12.         then Begin SL.Delete(I); end
  13.         else Begin Inc(I); end;
  14.     end;
  15.   SL.Free;
  16. end;

Bart

  • Hero Member
  • *****
  • Posts: 5516
    • Bart en Mariska's Webstek
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #1 on: March 11, 2020, 10:24:51 pm »
You'll have to loop from the last to the first line, otherwise you'll get an access violation.

Bart

jamie

  • Hero Member
  • *****
  • Posts: 6802
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #2 on: March 11, 2020, 10:26:42 pm »
Not directly but could be ways to speed up the process.
The only true wisdom is knowing you know nothing

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #3 on: March 11, 2020, 10:29:46 pm »
 @MaxCuriosus]

Hi - ongoing magician!

A text is a sequence of lines.
How should that work without a loop?

Loops and decisions are the very basic of a programming language,
the rest is "decoration".

Winni

eljo

  • Sr. Member
  • ****
  • Posts: 468
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #4 on: March 11, 2020, 11:00:22 pm »
In order to eliminate empty lines read from a text file, is there a way to do it without using an explicit loop as in the code below?

Code: Pascal  [Select][+][-]
  1. Var
  2.   SL: TStringList;
  3.   I: Integer;
  4.  
  5. Begin
  6.   SL:=TStringList.Create;
  7.   SL.LoadFromFile(SomeTextFile);
  8.   I:=0;
  9.   While I<SL.Count do
  10.     Begin
  11.       If SL.Strings[I]=''
  12.         then Begin SL.Delete(I); end
  13.         else Begin Inc(I); end;
  14.     end;
  15.   SL.Free;
  16. end;
depends on what you mean empty line. If a line has one or more spaces is it empty or not? If not then you can use the stringreplace function to replace a pair of lineend chars with a single lineend character eg
Code: Pascal  [Select][+][-]
  1.   sl.Text := StringReplace(sl.Text,LineEnding+LineEnding,LineEnding,[rfReplaceAll]);
  2.  
but stringreplace does have a character loop inside so...

You'll have to loop from the last to the first line, otherwise you'll get an access violation.

Bart
No he have not. He is using a while loop that increases the counter only if there is no deletion as such I doubt that either the sl.count or the i are buffered by the loop.

Bart

  • Hero Member
  • *****
  • Posts: 5516
    • Bart en Mariska's Webstek
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #5 on: March 11, 2020, 11:09:09 pm »
You'll have to loop from the last to the first line, otherwise you'll get an access violation.
No he have not. He is using a while loop that increases the counter only if there is no deletion as such I doubt that either the sl.count or the i are buffered by the loop.

You are correct, I didn't look at the code well enough.
A downto loop might be faste though (Count is only evaluated once in a for loop, but in the while loop it is evaluated on every loop).

Bart

jamie

  • Hero Member
  • *****
  • Posts: 6802
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #6 on: March 11, 2020, 11:18:33 pm »
That could be a slow way of doing things..

Create another StringList and load that one line by in the loop ignoring empty strings and delete the original stringlist

the other way would be to simply have two loop pointers, one reads while the other writes to the memory pointer of the stringlist. When you encoder a CRLF with no char count prior to then just skip it and move on otherwise keep writing the reads back using the WRITE pointer...

I can make an example if one wishes.
The only true wisdom is knowing you know nothing

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #7 on: March 11, 2020, 11:23:21 pm »
Hi!

Jamie - please dont make simple things complicated.

And Bart is right - for is faster than while .
And a downto is much more clear than deleting with a bottom-to-top loop.

Winni
« Last Edit: March 11, 2020, 11:34:12 pm by winni »

jamie

  • Hero Member
  • *****
  • Posts: 6802
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #8 on: March 12, 2020, 12:12:27 am »
I am not making them complicated..

Memory management is tricky and to work with large files it gets tricky to filter out items..

Delete lines means it needs to keep shuffling memory..

Please look at this one, this adds lines and since the way the memory manager works with the list it adds memory in chucks
Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. Var
  3.   S:tstringlist;
  4.   T:String;
  5.   CT:Integer;
  6. begin
  7.   S := TStringList.Create;
  8.   For CT := 0 To Memo1.Lines.Count-1 do
  9.    Begin
  10.      T := Memo1.Lines[CT];
  11.      if T <> '' Then S.Add(T);
  12.    end;
  13.  Memo1.Lines.Assign(S);
  14.  S.Free;
  15. end;                    
  16.  

I think you may find that to be faster on larger files
The only true wisdom is knowing you know nothing

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #9 on: March 12, 2020, 01:17:44 am »
Hi!

Then we can take a look at eljos idea.

Code: Pascal  [Select][+][-]
  1. procedure DeleteEmptyLines(var sl : TStr5ingList);
  2. var s : String;
  3.      p : integer;
  4.  
  5. begin
  6. s := sl.text;
  7. repeat
  8. p := pos (LineEnding+LineEnding,s);
  9. if p > 0 then delete (s,p,length(lineEnding));
  10. until p= 0;
  11. sl.text := s;
  12. end;

Winni

jamie

  • Hero Member
  • *****
  • Posts: 6802
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #10 on: March 12, 2020, 02:22:19 am »
Hi!

Then we can take a look at eljos idea.

Code: Pascal  [Select][+][-]
  1. procedure DeleteEmptyLines(var sl : TStr5ingList);
  2. var s : String;
  3.      p : integer;
  4.  
  5. begin
  6. s := sl.text;
  7. repeat
  8. p := pos (LineEnding+LineEnding,s);
  9. if p > 0 then delete (s,p,length(lineEnding));
  10. until p= 0;
  11. sl.text := s;
  12. end;

Winni

I guess if you wanted to remove all the lineEndings and have one long single line then use the above code because that is what's going to happen.
 Also that does exactly what I was talking about, shuffling memory on every instance found.

Code ideas are nice but they at least got to work to a useable level, speed is workable later.
The only true wisdom is knowing you know nothing

PascalDragon

  • Hero Member
  • *****
  • Posts: 5870
  • Compiler Developer
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #11 on: March 12, 2020, 09:37:30 am »
Memory management is tricky and to work with large files it gets tricky to filter out items..

Delete lines means it needs to keep shuffling memory..

And duplicating the list means that the array of strings inside the string list is allocated twice (while the strings should exist only once due to reference counting). That is something one needs to keep in mind as well.

MoCityMM

  • Jr. Member
  • **
  • Posts: 72
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #12 on: March 12, 2020, 10:03:42 am »
If I am reading this correctly (4am my time, no coffee yet)...

You want to eliminate the empty string(s) in a TStringList loaded from file?

I used the following in my current side project and it works, there are 3 rows:

for CTAHStrListRow := CTAHistoryStrList.Count - 1 downto 0 do
               
                    begin

                         if Trim(CTAHistoryStrList[CTAHStrListRow]) = '' then

                            begin

                                 CTAHistoryStrList.Delete(CTAHStrListRow);

                            end;

                     end;

test it see if it works for you (change the naming convention to fit your needs) and if I didn't read it correctly, ignore it.

Regards,
-Mo                           

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #13 on: March 12, 2020, 10:53:11 am »
Hi!

TStringlist.delete only needs an integer as parameter!

Code: Pascal  [Select][+][-]
  1. for i := CTAHistoryStrList.Count - 1 downto 0 do        
  2.     begin
  3.          if Trim(CTAHistoryStrList[i]) = '' then
  4.                                  CTAHistoryStrList.Delete(i);
  5.          end; // for
  6.  
And format your code here, please.

Winni

MoCityMM

  • Jr. Member
  • **
  • Posts: 72
Re: Eliminating empty strings in TStringList when reading from text file.
« Reply #14 on: March 12, 2020, 11:00:19 am »
Morning Winni,

I am verbose with my coding 'CTAHStrListRow' is a integer variable.  :D

And I am deleting more than just empty strings from my TStringList, thus the extra begin and end included in sample.

-Mo
« Last Edit: March 12, 2020, 11:26:56 am by MoCityMM »

 

TinyPortal © 2005-2018