Recent

Author Topic: Parsing STDERR / STDOUT  (Read 696 times)

pcurtis

  • Hero Member
  • *****
  • Posts: 634
Parsing STDERR / STDOUT
« on: June 04, 2021, 06:52:40 pm »
I use the following code to create a process (in a thread)

Code: Pascal  [Select][+][-]
  1.   MyShell := TProcess.Create(nil);
  2.     with MyShell do
  3.       begin
  4.         Options := [poUsePipes];
  5.         ShowWindow := swoHIDE;
  6.         Executable := 'D:\Lazarus\Shared\mpv.exe';
  7.         Options := Options + [poUsePipes];
  8.         Parameters.Add('--wid=' + inttostr(WID));
  9.         while not MyQuery.EOF do
  10.            begin
  11.              Parameters.Add(MyQuery['fnPARAMETER'] + MyQuery['fnVALUE']);
  12.              MyQuery.Next;
  13.            end;
  14.         Parameters.Add(ClipFilename);
  15.         Execute;
  16.       end;
  17.  
  18.     while (MyShell.Running) or
  19.           (MyShell.Stderr.NumBytesAvailable > 0) do
  20.       begin
  21.         while MyShell.Stderr.NumBytesAvailable > 0 do
  22.           begin
  23.             fPlayVideoStatus := '';
  24.             begin
  25.               ReadCount := Min(512, MyShell.Stderr.NumBytesAvailable); //Read up to buffer, not more
  26.               MyShell.Stderr.Read(CharBuffer, ReadCount);
  27.  
  28.               fPlayVideoStatus := fPlayVideoStatus + CharBuffer;
  29.             end;
  30.               fWhere := 1;
  31.               Synchronize(@PlayVideoStatus);
  32.           end;
  33.  
  34.         while MyShell.Output.NumBytesAvailable > 0 do
  35.           begin
  36.             fPlayVideoStatus := '';
  37.               begin
  38.                 ReadCount := Min(512, MyShell.Output.NumBytesAvailable); //Read up to buffer, not more
  39.                 MyShell.Output.Read(CharBuffer, ReadCount);
  40.  
  41.                 fPlayVideoStatus := fPlayVideoStatus + CharBuffer;
  42.               end;
  43.  
  44.             fWhere := 2;
  45.             Synchronize(@PlayVideoStatus);
  46.           end;
  47.  
  48.  

All works fine except parsing STDERR / STDOUT.

I would like to  Synchronize(@PlayVideoStatus) STDERR / STDOUT line by line (the sync code is fine). The problem really lyes in reading STDERR / STDOUT and parsing it.

I know @tedson has UnTerminal-1.0 that I think will do the trick, but I don't know how to integrate it.
Can anyone help?
Windows 10
Laz 2.0.10
FPC 3.2.0

jamie

  • Hero Member
  • *****
  • Posts: 4732
Re: Parsing STDERR / STDOUT
« Reply #1 on: June 04, 2021, 08:18:55 pm »
What are you talking about, splitting lines when line endings appears and then generate a line of out as a string ?
The only true wisdom is knowing you know nothing

pcurtis

  • Hero Member
  • *****
  • Posts: 634
Re: Parsing STDERR / STDOUT
« Reply #2 on: June 04, 2021, 08:47:41 pm »
So for example STDERR is read in 512 byte blocks. I want to break this into CRLF terminated strings. Data is constantly appearing on STDERR
Windows 10
Laz 2.0.10
FPC 3.2.0

jamie

  • Hero Member
  • *****
  • Posts: 4732
Re: Parsing STDERR / STDOUT
« Reply #3 on: June 04, 2021, 09:32:24 pm »
Use string routines

MyDataString

When you start set it to '';

in the loop just treat the 512 buffer has array of char and add a 0 at the end of it each time to ensure the routines work..

MyDataString := MyDataString+Pchar(@My512Buffer)^;

empty the My512 for next read.

If Pos(#13, MyDataString) <> 0 Then
 
 Finalstring := Copy(…….) etc...

you know the rest...
I can give you example code block later...


The only true wisdom is knowing you know nothing

jamie

  • Hero Member
  • *****
  • Posts: 4732
Re: Parsing STDERR / STDOUT
« Reply #4 on: June 04, 2021, 10:40:30 pm »
Code: Pascal  [Select][+][-]
  1. var
  2.   Form1: TForm1;
  3.   MyBufferList:TstringList; //Create this at start somewhere., keep it alive
  4.  
  5. implementation
  6.  
  7. {$R *.lfm}
  8.  
  9. { TForm1 }
  10. Function StrFromMyBuf(Var theBuf:Array of Char;Count:Integer):TStringList;
  11. Begin
  12.   Result := MyBufferList;
  13.   If Count = 0 Then exit;
  14.   Result.SkipLastLineBreak :=True;
  15.   Result.Text := Result.Text+Copy(Pchar(@Thebuf)^,0,Count-1);
  16.   If Result.Count > 100 Then Result.Delete(0); //Keep list limited;
  17. end;
  18.  

Something like that.
that should give you something to work with.
make sure you empty the buffer each time you call this..

The only true wisdom is knowing you know nothing

jamie

  • Hero Member
  • *****
  • Posts: 4732
Re: Parsing STDERR / STDOUT
« Reply #5 on: June 05, 2021, 12:23:21 am »
The last one won't work because for some reason the array isn't being past around correctly...

I did a local test with this one.

I am assuming because its a network thing that #10 will be the predominate line ending here but it will check for CR in case it gets in there.
Basically because if only a #13 is received then you can switch it around.. I suppose you could do a test to determine which order is being use..

This version requires that you maintain two items somewhere between calls.

Code: Pascal  [Select][+][-]
  1. var
  2.   Form1: TForm1;
  3.   MyBufferList:TstringList; //Create this at start somewhere., keep it alive
  4.   WorkString :String;
  5. implementation
  6.  
  7. {$R *.lfm}
  8.  
  9. { TForm1 }
  10. Function StrFromMyBuf(TheBuf:Pchar;Count:Integer):TStringList;
  11. var
  12.   E:Pchar;
  13.   L:Integer;
  14.   LE:Char;
  15. Begin
  16.   E := TheBuf; Inc(E,Count); //Mark the buffer end;
  17.   L:= 0;
  18.   While TheBuf < E DO
  19.    Begin
  20.     if TheBUf^ <> #10 Then WorkString := WorkString+TheBuf^ Else
  21.      Begin
  22.        While Pos(#13, WorkString)<>0 DO Delete(WorkString, Pos(#13,WorkString),1);
  23.        MyBufferList.Add(WorkString);
  24.        WorkString := '';
  25.      end;
  26.     Inc(TheBuf);
  27.    end;
  28. end;
  29.                          
  30.  

Cast your buffer to a PChar(@MyBuffer), Count etc..
The only true wisdom is knowing you know nothing

jamie

  • Hero Member
  • *****
  • Posts: 4732
Re: Parsing STDERR / STDOUT
« Reply #6 on: June 05, 2021, 03:42:43 pm »
Here is an example project using a class I wrote that can auto detect on the fly or line endings since I've found files that mix the line endings.

  This code is part of another project I did, I stripped out some stuff to keep it simple here but I ran into problems reading different files from sources of mixed line endings.

 Some had #13#10, others had only #10 or #3 and some had #10#13 etc..

In your case I stripped it down but you can add a TCriticalSection in the block read to protect it from multiple threads using the same instance..

Attached is a project..
The only true wisdom is knowing you know nothing

 

TinyPortal © 2005-2018