Recent

Author Topic: Having trouble with TCSVParser  (Read 586 times)

dcom67

  • New member
  • *
  • Posts: 8
Having trouble with TCSVParser
« on: August 14, 2020, 08:33:26 pm »
The parser is skipping to the end of the string. It worked for a while and now it has stopped working

FM.LB7.Items.Strings[e]  = 2,8,7,0,0,-1,-1,1,MNLY:1,11,3:0,8:0,3:1,8:0,3:2,8:0,3:3,8:0,3:4,8:0,3:5

and it is stopping here  pTM^.House with an e convert error  not a valid Integer  and the  "S" value is showing  as " 3:5"  so it has skipped everything in the string

Maybe some one with eyes wider than mine can see why :-)

Parser:=TCSVParser.Create;
                     Parser.Delimiter:=',';
                     for e:= 0 to FM.LB7.Items.Count -1 do
                     begin
                          Parser.SetSource(FM.LB7.Items.Strings[e]);
                          s := '';
                          pTM := RATeamL.NewRATeamAdd;
                          Parser.ParseNextCell;
                          s :=  Parser.CurrentCellText;
                          pTM^.House:= StrToInt(s);
                          Parser.ParseNextCell;
                          s :=  Parser.CurrentCellText;
                          pTM^.TType1:= StrToInt(s);
                          Parser.ParseNextCell;
                          s :=  Parser.CurrentCellText;
                          pTM^.Priority:= StrToInt(s); 

                         ~~~~~~~~

                      end;
                     Parser.Free;

wp

  • Hero Member
  • *****
  • Posts: 7551
Re: Having trouble with TCSVParser
« Reply #1 on: August 14, 2020, 10:12:26 pm »
I condensed your snippet to something compilable, and it is working fine... So, the issue must be somewhere else. Step through your code with the debugger and check whether the variables are as expected.

What is your Lazarus/FPC version, your OS? Which bitness (32 bit or 64 bit)?

Code: Pascal  [Select][+][-]
  1. program Project1;
  2. uses
  3.   SysUtils, csvreadwrite;
  4. const
  5.   str: String = '2,8,7,0,0,-1,-1,1,MNLY:1,11,3:0,8:0,3:1,8:0,3:2,8:0,3:3,8:0,3:4,8:0,3:5';
  6. var
  7.   parser: TCSVParser;
  8.   s: String;
  9. begin
  10.   parser := TCSVParser.Create;
  11.   Parser.Delimiter:=',';
  12.   Parser.SetSource(str);
  13.  
  14.   s := '';
  15.   Parser.ParseNextCell;
  16.  
  17.   s :=  Parser.CurrentCellText;
  18.   WriteLn(s, ', ', StrToInt(s));
  19.  
  20.   Parser.ParseNextCell;
  21.   s :=  Parser.CurrentCellText;
  22.   WriteLn(s, ', ', StrToInt(s));
  23.  
  24.   Parser.ParseNextCell;
  25.   s :=  Parser.CurrentCellText;
  26.   WriteLn(s, ', ', StrToInt(s));
  27.  
  28.   parser.Free;
  29.   ReadLn;
  30. end.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

dcom67

  • New member
  • *
  • Posts: 8
Re: Having trouble with TCSVParser
« Reply #2 on: August 15, 2020, 06:03:30 am »
I'm on win 10  64 bit
And it was a fairly old Lazarus but I just upgraded it today.   ver 2.0.10  and the fpc is 3.2.0   and that didn't fix it.

 I did step through the code that how i know where it is stopping, and the values.
The part i don't understand it was working then all of a sudden it quit.

I did find out how to get it to work . i free the parser and create a new one inside of the for loop , so i make and free a new parser for each line

still don't understand why it does that it would read the first line fine then stop on the second one,''

And i have code further down that does it the way i did previously  just create the parse once , and set the source line in the for loop  and it works fine.
« Last Edit: August 15, 2020, 06:49:05 am by dcom67 »

wp

  • Hero Member
  • *****
  • Posts: 7551
Re: Having trouble with TCSVParser
« Reply #3 on: August 15, 2020, 11:26:27 am »
I did find out how to get it to work . i free the parser and create a new one inside of the for loop , so i make and free a new parser for each line
This sounds like a missing reset of the parser (parser.ResetParser), but this is called automatically when a new source string is assigned by parser.SetSource().

What is happening in the '~~~~~~~' part of the posted code?

You should try to extract a little (compilable) demo project from the failing part of your program. Pack all source files (*.pas, *.lpi, *.lpr, *.lfm, but no .exe, no .ppu etc) to a common zip and upload it here under "Attachments and other options"). Of course, the demo project must show the error. Without code it is not possible to say what is wrong here.
« Last Edit: August 15, 2020, 12:52:21 pm by wp »
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

dcom67

  • New member
  • *
  • Posts: 8
Re: Having trouble with TCSVParser
« Reply #4 on: August 15, 2020, 04:00:52 pm »
Here ill post the entire function, ********* Edited Out   *************
It will take me a while to separate it into a smaller package

It fails in the TeamTypes  , The Trigs section it works fine , Trigs was the first section i wrote

 




               
« Last Edit: August 15, 2020, 06:14:26 pm by dcom67 »

wp

  • Hero Member
  • *****
  • Posts: 7551
Re: Having trouble with TCSVParser
« Reply #5 on: August 15, 2020, 05:14:05 pm »
Hey come on - I was asking for something what I can compile and debug myself. I cannot do this with your code fragment. I hope you understand that I do not want to spend part of my my weekend to create a compilable project out of this. Moveover I'll have to add modifications because I do not know what pTM, FM.LBTeams, .LBTeam1, LB6, LB7 are. With every modification that I make the chance increases that the program will become functional...

So please: Create a new project and add from your main project only what is needed to show the bug. Compile and run this demo project. Upload it only when it shows the error. Do not paste code snippets into posts, they are useless for real help (and if you do enclose the code by [code]...[/code] tags to activate the syntax highlighter and to avoid effects like suddenly setting the code to bold face because there is a [b] in your text.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

jamie

  • Hero Member
  • *****
  • Posts: 3532
Re: Having trouble with TCSVParser
« Reply #6 on: August 15, 2020, 05:56:56 pm »
I guess a good question would be is, who's TCSVParser is he using ?

I can find several TCSVParsers all over the net in source code all being different..

@WP, you are using CSVReadWrite unit, what is he using ? I don't see that reference unless I've gone blind ?
The only true wisdom is knowing you know nothing

dcom67

  • New member
  • *
  • Posts: 8
Re: Having trouble with TCSVParser
« Reply #7 on: August 15, 2020, 06:13:07 pm »
OK Sorry for pasting all of that , I thought you wanted to know what was after the ~~~~~~~~~~~

Ok i got it into a small project. It should fail if you try to open the included *.Mpr file

don't worry about the global or mainform. pas the trouble is in the trigs unit.  I have commented out the parts to change for it to work .  Right now it fails . after skipping the line text

Oh and it uses  CSVDocument  from the fpc i presume , what ever comes with lazarus

wp

  • Hero Member
  • *****
  • Posts: 7551
Re: Having trouble with TCSVParser
« Reply #8 on: August 15, 2020, 06:33:47 pm »
dcom67, excuse my angry words.

In the meantime I looked at your code again and was able to extract a compilable project from your posted fragement. And it does show the error reported in the second loop iteration when I activate the parser used inside the loop.

See attached demo.

The problem is: When the Parser.SetSource is called for the second time the internal string stream for the string to be analyzed happens to be allocated at the same memory position as in the first run. Since SetSource checks whether the old and new streams are "equal" (memory location!) this method is exited immediately before doing anything else, in particular ignoring the ResetParser method which prepares the parser for a new stream to be analyzed. Without this the parser is still at the end of the previous string and finds the non-integer '3:5' string. Therefore, you simply have to call ResetParser by yourself after SetSource() to solve your problem:

Code: Pascal  [Select][+][-]
  1.   Parser:=TCSVParser.Create;
  2.   Parser.Delimiter:=',';
  3.   for e:= 0 to LB7.Items.Count -1 do
  4.   begin
  5.     Parser.SetSource(LB7.Items.Strings[e]);
  6.     Parser.ResetParser;
  7.     [...]

Note:
I copied the csvreadwrite unit from its fpc folder to the project folder because when the unit is not used by other FPC units needed by the program the compiler will use this copy instead of the orginal version. This way it is easy to step into an FCL unit with the debugger.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

wp

  • Hero Member
  • *****
  • Posts: 7551
Re: Having trouble with TCSVParser
« Reply #9 on: August 15, 2020, 06:42:35 pm »
I forgot to add this:

Since your csv lines are rather simple (meaning: no line breaks within cells, no quoted substrings) you could also use another StringList to do the csv decoding. There is a property CommaText which splits a string at commas and puts the parts into the elements of the StringList. So, to be precise, here is some alternative code, without csvreadwrite and TCSVParser (untested):

Code: Pascal  [Select][+][-]
  1. var
  2.   FieldList: TStringList;
  3. ...
  4.   FieldList := TStringList.Create;
  5.   for e:= 0 to LB7.Items.Count -1 do  
  6.   begin
  7.     FieldList.CommaText := LB7.Items.Strings[e];
  8.     pTM := RATeamL.NewRATeamAdd;  
  9.     pTM^.House:= StrToInt(FieldList[0]);
  10.     pTM^.TType1 := StrToInt(FieldList[1]);
  11.     pTM^.Priority := StrToInt(FieldList[2]);
  12.     ...
  13.   end;
  14.   FieldList.Free;

And since fpc3.0.x there is a further solution avoiding the auxiliary stringlist and using a string array. It is based on the .Split helper function available to every string:

Code: Pascal  [Select][+][-]
  1. var
  2.   sa: TStringArray;
  3. ...
  4.   for e:= 0 to LB7.Items.Count -1 do  
  5.   begin
  6.     sa := LB7.Items.Strings[e]. Split(',');  // <--- split the string at the commas
  7.     pTM := RATeamL.NewRATeamAdd;  
  8.     pTM^.House:= StrToInt(sa[0]);
  9.     pTM^.TType1 := StrToInt(sa[1]);
  10.     pTM^.Priority := StrToInt(sa[2]);
  11.     ...
  12.   end;

« Last Edit: August 15, 2020, 06:45:05 pm by wp »
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

dcom67

  • New member
  • *
  • Posts: 8
Re: Having trouble with TCSVParser
« Reply #10 on: August 15, 2020, 07:01:31 pm »
Wow your fast  :D

And thanks for the tips and your effort to solve this .   I will use the reset parser method for now , I will work on making the code simpler later.

And It works Yah Hoo :)
« Last Edit: August 15, 2020, 07:15:26 pm by dcom67 »

 

TinyPortal © 2005-2018