Tstringlist

[scons]

Hello,

I am a little bit stuck at this following challenge, at least for me it is a challenge:

I am reading a textfile into a Tstringlist. At some point in this list (line nr 27) the files ends, or goes on with extra code. The extra code can be something like this:

Code: Text  [Select][+][-]

1. BO
2.   v   55.00u   90.00   18.00
3.   v   55.00u   30.00   18.00
4.   v  155.00u   90.00   18.00
5.   v  155.00u   30.00   18.00
6.   v  255.00u   90.00   18.00
7.   v  255.00u   30.00   18.00
8. EN

There can be several "blocks" like this one. BO = start of the block, EN is end of the block and/or end of the textfile (there is always an empty line at the bottom of the textfile after the last EN).

You can see that the content of these block(s) is always the same structure: a letter an then 3 numbers with or without an extension letter to the number.
I need to work with these data (numbers + letters). What is the most efficient way to do this ? I am assuming the result of the Tstringlis.Count line is:

Code: Text  [Select][+][-]

1.   v   55.00u   30.00   18.00

Is the best way to break up each Tstringlist line ? And how do I make a loop the read each block ?

Some more block examples:

Code: Text  [Select][+][-]

1. BO
2.   v 3545.00s   57.00   14.00
3.   v 3635.00s   57.00   14.00
4.   o 3440.00s   90.00   18.00
5.   o 3440.00s   30.00   18.00
6.   o 3540.00s   90.00   18.00
7.   o 3540.00s   30.00   18.00
8.   o 3640.00s   90.00   18.00
9.   o 3640.00s   30.00   18.00
10. EN

Code: Text  [Select][+][-]

1. BO
2.   v 3387.50o  100.00   14.00
3.   v 3477.50o  100.00   14.00
4. EN
5.  

Thanks

[lainz]

Code: Pascal  [Select][+][-]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.   Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs;
9.  
10. type
11.  
12.   { TForm1 }
13.  
14.   TForm1 = class(TForm)
15.     procedure FormCreate(Sender: TObject);
16.   private
17.     { private declarations }
18.   public
19.     { public declarations }
20.   end;
21.  
22. var
23.   Form1: TForm1;
24.  
25. implementation
26.  
27. uses
28.   LazUTF8;
29.  
30. {$R *.lfm}
31.  
32. { TForm1 }
33.  
34. procedure TForm1.FormCreate(Sender: TObject);
35. var
36.   i: integer;
37.   nextline: boolean;
38.   sfile, sline: TStringList;
39. begin
40.   sfile := TStringList.Create;
41.   sfile.LoadFromFile('file.txt');
42.  
43.   sline := TStringList.Create;
44.   for i := 0 to sfile.Count - 1 do
45.   begin
46.     case UTF8LowerCase(sfile[i]) of
47.       'bo': nextline := True;
48.       'en': nextline := False;
49.       else
50.       begin
51.         if nextline then
52.         begin
53.           sline.CommaText := sfile[i];
54.           // do what you want wit it
55.           writeln(sline.Text);
56.         end;
57.       end;
58.     end;
59.   end;
60.  
61.   sline.Free;
62.   sfile.Free;
63. end;
64.  
65. end.

[molly]

I am assuming the result of the Tstringlis.Count line is:
Is the best way to break up each Tstringlist line ? And how do I make a loop the read each block ?

Yes. Just use a integer variable to 'loop' through your individual lines in the stringlist, comparing against the count property.

- Read current stringlist item into a string
- if string starts with BO then subloop until EN is found
- for every item between BO and EN, 'break' the string into individual items with something like extractword()
- Do something with the extracted data, e.g. convert into a record declaring each individual record field as floating point value (assuming that you require as such).

Just start with the things that you are able to accomplish and then show us some code. E.g. there is hardly any learning moment for you if you do not know exactly where you get stuck.

More than happy to supply a working example but there comes a day you need to analyze these things yourself and break them up into smaller pieces that you are able to solve problems like this yourself.

Besides that, it is not 100% clear if it is enough by simply just parsing and directly using the values or that you need to access the data later as well (in which case i would suggest using a list of records).

E.g., that requires two different approaches even though extraction of the data from the stringlist can look similar.

[Thaddy]

This is an FPC question not a Lazarus one.
Why the UTF8LowerCase?
Why doesn't it work? But close enough, I repaired it for fpc:
This works. It's probably an old cobol file or something.

Code: Pascal  [Select][+][-]

1. program untitled;
2. {$ifdef fpc}{$mode objfpc}{$endif}
3. uses classes,sysutils;
4. var
5.   i: integer;
6.   nextline: boolean;
7.   sfile, sline: TStringList;
8. begin
9.   sfile := TStringList.Create;
10.   if fileExists('file.txt') then
11.   try
12.     sfile.LoadFromFile('file.txt');
13.     sline := TStringList.Create;
14.     try
15.       for i := 0 {or 27 if you are sure about 27}  to sfile.Count - 1 do
16.         case sfile[i] of
17.         'BO': nextline := True;
18.         'EN': nextline := False;
19.         else
20.         begin
21.           if nextline then
22.           begin
23.             sline.CommaText := sfile[i];
24.             // do what you want wit it
25.             writeln(sline.Text);
26.           end;
27.          end;
28.        end
29.      finally
30.        sline.free;
31.     end;
32.   finally
33.     sfile.free;
34.   end;
35. end.
36.  

[lainz]

I like the lower case, if you're sure it is always upper case go ahead.

[Thaddy]

I like the lower case, if you're sure it is always upper case go ahead.

For my code: yes,plz. No BASIC.
For my data: no, plz, I want to process data efficiently. The lowercase function you used is - very - expensive and not warranted in this case.

[lainz]

I like the lower case, if you're sure it is always upper case go ahead.

For my code: yes,plz. No BASIC.
For my data: no, plz, I want to process data efficiently. The lowercase function you used is - very - expensive and not warranted in this case.

Good point, if you can trust the data of course is not neccesary.

[howardpc]

@scons
Here's an example for you that uses a dynamic array of advanced records rather than a class to store the parsed data.

[scons]

To lainz, molly, Thaddy and howardpc : thank you guys for showing me into the right direction. Very much appreciated !!

@ lainz/Thaddy: I'm exploring this piece of code. Works pretty good, I just need to figure out how to separate the letter from the numbers now and then I can do a testrun.
 

Code: Text  [Select][+][-]

1. 55.00u -> 55.00 and u

@ howardpc: wow dude, that is some code ! I see what it does, but I will try to finish the first solution before I start deeper exploring yours.

[molly]

I just need to figure out how to separate the letter from the numbers now and then I can do a testrun.

The routine i used to put the string into a record:

Code: Pascal  [Select][+][-]

1. function StringToRecord(S: String): TBlockItem;
2. var
3.   i,
4.   n : integer;
5.   w : String;
6.   c : Word = 0;
7. begin
8.   // words are separated by spaces
9.   n := WordCount(S, [' ']);
10.  
11.   for i := 1 to n do
12.   begin
13.     w := ExtractWord(i, S, [' ']);
14.     if (i = 1) then Result.Prefix := W[1];
15.  
16.     if (i = 2) then
17.     begin
18.       Result.Suffix := w[Length(w)];
19.       Delete(w, Length(w), 1);
20.       Val(w, Result.Value1, c);
21.     end;
22.    
23.     if (i = 3) then Val(w, Result.Value2, c);
24.  
25.     if (i = 4) then Val(w, Result.Value3, c);
26.   end;
27. end;
28.  

Where S is a line in a block and TBlockItem reads:

Code: Pascal  [Select][+][-]

1. type
2.   TBlockItem = record
3.     Prefix  : Char;
4.     Value1  : Float;
5.     Suffix  : Char;
6.     Value2  : Float;
7.     Value3  : Float;
8.   end;
9.  

[lainz]

Other way:

Code: Pascal  [Select][+][-]

1. var
2.   i: integer;
3.   nextline: boolean;
4.   sfile, sline: TStringList;
5.   s, t: string;
6.   l: integer;
7. begin
8.   sfile := TStringList.Create;
9.   sfile.LoadFromFile('file.txt');
10.  
11.   sline := TStringList.Create;
12.   for i := 0 to sfile.Count - 1 do
13.   begin
14.     case sfile[i] of
15.       'BO': nextline := True;
16.       'EN': nextline := False;
17.       else
18.       begin
19.         if nextline then
20.         begin
21.           sline.CommaText := sfile[i];
22.  
23.           // s is number and t is suffix
24.           l := UTF8Length(sline[1]);
25.           s := UTF8Copy(sline[1], 1, l-1);
26.           t := UTF8Copy(sline[1], l, 1);
27.           writeln(s, ' and ', t);
28.  
29.           // do what you want wit it
30.           //writeln(sline.Text);
31.  
32.         end;
33.       end;
34.     end;
35.   end;
36.  
37.   sline.Free;
38.   sfile.Free;

[scons]

Thanks again guys.

I am trying lianz his code first, since I was already playing around with his example.

I did a throw at it like this:

Code: Pascal  [Select][+][-]

1.  procedure TForm1.ListBox1Click(Sender: TObject);
2. var
3.   i, o, error: integer;
4.   numX, numY,holX, holY, radX, scale: real;
5.   varX, varY: string;
6.   test : double;
7.   X, Y, varX1, varY1, holX1, holY1, radX1: single;
8.   NCList, sline: TStringList;
9.   varprof: Ansistring;
10.   nextline: boolean;
11.   bmp: TBGRABitmap;
12.   c: TBGRAPixel;
13. begin
14.   i := ListBox1.ItemIndex;
15.   if i < 0 then
16.   begin
17.     ProfileLabel.Caption := '';
18.     ProfileLabel.Visible := False;
19.     AmountLabel.Caption := '';
20.     AmountLabel.Visible := False;
21.     SteelQualLabel.Caption := '';
22.     SteelQualLabel.Visible := False;
23.     LengthLabel.Caption := '';
24.     LengthLabel.Visible := False;
25.     WidthLabel.Caption := '';
26.     WidthLabel.Visible := False;
27.     Exit;       // -1 means there is no selected item
28.   end;
29.   try
30.   ProjectNrLabel.Visible := True;
31.   ProfileLabel.Visible := True;
32.   AmountLabel.Visible := True;
33.   SteelQualLabel.Visible := True;
34.   LengthLabel.Visible := True;
35.   WidthText.Visible := True;
36.   WidthLabel.Visible := True;
37.   NCList := TStringList.Create;
38.   sline := TStringList.Create;
39.   NCList.LoadFromFile(ListBox1.Items[i]);
40.   if NCList.Count >= 3 then
41.     ProjectNrLabel.Caption := Trim(NCList[2]);
42.   if NCList.Count >= 7 then
43.     SteelQualLabel.Caption := Trim(NCList[6]);
44.   if NCList.Count >= 8 then
45.     AmountLabel.Caption := Trim(NCList[7]);
46.   if NCList.Count >= 9 then
47.     ProfileLabel.Caption := Trim(NCList[8]);
48.   if NCList.Count >= 11 then
49.   varX := NCList[10];
50.   if NCList.Count >= 12 then
51.    varY := NCList[11];
52.   if NCList.Count >= 11 then
53.    LengthLabel.Caption := Trim(NCList[10]);
54.   if NCList.Count >= 13 then
55.     WidthLabel.Caption := Trim(NCList[12]);
56.   //
57.   Val(varX,numX,error);
58.   Val(varY,numY,error);
59.   VarX1:=numX+20;
60.   VarY1:=numY+20;
61.   PaintBox1.Canvas.Clear();
62.   bmp := TBGRABitmap.Create(PaintBox1.Width, PaintBox1.Height,BGRAWhite);
63.   c := ColorToBGRA(ColorToRGB(clWindowText));
64.   bmp.JoinStyle := pjsRound;
65.   X:= (PaintBox1.Width / 2);
66.   Y:= (PaintBox1.Height / 2);
67.  
68.   if NCList.Count >= 10 then
69.   begin
70.   varprof := Trim(NCList[9]);
71.   end;
72.   // I selection to bitmap
73.   if varprof = ('I') then
74.   begin
75.   if VarX1 > (X*2+30) then
76.   begin
77.     scale := VarX1 / (X*2);
78.     varX1 := VarX1 / scale;
79.     VarY1 := VarY1 / scale;
80.   end;
81.   bmp.LineCap := pecSquare;
82.   bmp.PenStyle := psSolid;
83.   bmp.DrawPolylineAntialias([PointF(10,50), PointF(VarX1,50), PointF(VarX1,VarY1), PointF(10,VarY1), PointF(10,50)],c,3);
84.   bmp.Draw(PaintBox1.Canvas,0,0,False);
85.   bmp.Free;
86.   end;
87.    // B selection to bitmap
88.   if varprof = ('B') then
89.   begin
90.     WidthLabel.Visible := False;
91.     WidthText.Visible := False;
92.  
93.   if VarX1 > (X*2+30) then
94.   begin
95.     scale := VarX1 / (X*2);
96.     varX1 := VarX1 / scale;
97.     VarY1 := VarY1 / scale;
98.   end;
99.  
100.   try
101.     for o := 27 {or 27 if you are sure about 27}  to NCList.Count - 1 do
102.       case NCList[o] of
103.       'BO': nextline := True;
104.       'EN': nextline := False;
105.       else
106.       begin
107.         if nextline then
108.         begin
109.           sline.CommaText := NCList[o];
110.           // do what you want with it
111.  
112.           Val(sline.ValueFromIndex[1],holX,error);
113.           Val(sline.ValueFromIndex[2],holY,error);
114.           Val(sline.ValueFromIndex[3],radX,error);
115.           holX1 := holX;
116.           holY1 := holY;
117.           radX1 := radX/2;
118.           //test := (holX1);
119.  
120.           bmp.LineCap := pecSquare;
121.           bmp.PenStyle := psSolid;
122.           bmp.EllipseAntialias(holX1,holY1,radX1,radX1,c,3);
123.         end;
124.        end;
125.      end
126.    finally
127.      sline.free;
128.   end;
129.  
130.   bmp.LineCap := pecSquare;
131.   bmp.PenStyle := psSolid;
132.   bmp.DrawPolylineAntialias([PointF(10,50), PointF(VarX1,50), PointF(VarX1,VarY1), PointF(10,VarY1), PointF(10,50)],c,3);
133.   bmp.Draw(PaintBox1.Canvas,0,0,False);
134.   bmp.Free;
135.   end;
136.   //ShowMessage('Error');
137.   finally
138.     if Assigned(NCList) then
139.     FreeAndNil(NCList);
140.   end;
141. end;
142.  

This results to this: fileviewer-01.png
But it should be this : fileviewer-02.png

The X-value is off (=0) due to the fact that the first value is number+letter = 0.00
That is the reason why I want to separate numbers and letters.

So the first number is an X-value, the second number is an Y-value and the third number is a radius. The letters represent extra symbols, or non if there is no letter extension.

By adding this code from lainz, it seems it is going not into the loop anymore:

Code: Pascal  [Select][+][-]

1.   for i := 0 to sfile.Count - 1 do
2.   begin
3.     case sfile[i] of
4.       'BO': nextline := True;
5.       'EN': nextline := False;
6.       else
7.       begin
8.         if nextline then
9.         begin
10.           sline.CommaText := sfile[i];
11.  
12.           // s is number and t is suffix
13.           l := UTF8Length(sline[1]);
14.           s := UTF8Copy(sline[1], 1, l-1);
15.           t := UTF8Copy(sline[1], l, 1);
16.           writeln(s, ' and ', t);
17.  
18.           // do what you want wit it
19.           //writeln(sline.Text);
20.  
21.         end;
22.       end;
23.     end;
24.   end;

[Thaddy]

I may have put you on the wrong path and introduced an off-by one error. Try start at 26 instead of 27. On which line does the first block start? That should bbe the first index if the fileheader is always the same.
Also: your files seem to be in ANSI format. If that's the case, declare all string variables concerned with reading and processing the file as AnsiString and use normal (ansi) string manipulation functions instead of the utf8 ones. Should be much faster.

[scons]

Yes this did the trick, thanks