Lazarus

Programming => Operating Systems => Linux => Topic started by: ThomasK on June 09, 2021, 11:36:53 am

Title: Piping Console Output into GUI Application
Post by: ThomasK on June 09, 2021, 11:36:53 am
I need some inspiration...

Is it possibe to pipe the console output of a GUI application into this application itself?

I attach the GUI itself and the Console Input/Output of the debugger.

The question is now, can I somehow get the Console Output istelf to show it to the user without debugger?


Thanks Thomas
Title: Re: Piping Console Output into GUI Application
Post by: marcov on June 09, 2021, 11:44:24 am

Close(output);
stdoutstream:=TMemoryStream.Create;
streamio.assignstream(output,stdoutstream);

Now using a timer, poll stdoutstream to see if something is changed, and append it to the memo.
Title: Re: Piping Console Output into GUI Application
Post by: MarkMLl on June 09, 2021, 12:26:51 pm
Quote from: ThomasK on June 09, 2021, 11:36:53 am
Is it possibe to pipe the console output of a GUI application into this application itself?

Tell us more about what /exactly/ you are trying to do, what OS you're running etc.

MarkMLl
Title: Re: Piping Console Output into GUI Application
Post by: ThomasK on June 09, 2021, 12:50:02 pm
I am posting in the Linux Part of the forum-> os is Linux, or better Raspbian.
I run a GUI application, when calls receive a -1 as return, error messages of the called underlying os processes are posted on the console.
I can see that the Lazarus debugging session in the console window of the debugger.
I want to be able to show those messages in my application.
Without debugger.
Title: Re: Piping Console Output into GUI Application
Post by: MarkMLl on June 09, 2021, 01:43:54 pm
Quote from: ThomasK on June 09, 2021, 12:50:02 pm
I am posting in the Linux Part of the forum->

Your message appeared in the "Recent" pane, and I've got more pressing things to do than check its provenance.

Like helping.

Quote
os is Linux, or better Raspbian.
I run a GUI application, when calls receive a -1 as return, error messages of the called underlying os processes are posted on the console.
I can see that the Lazarus debugging session in the console window of the debugger.
I want to be able to show those messages in my application.
Without debugger.

I presume that when you say "console" you mean a shell session, rather than the physical serial console or the kernel log.

The first thing you need to check is whether the message is going to stdout or stderr.

I've definitely been able to run a shell session as an external process and capture its output to go into a terminal emulator session which existed as a pane on a program's form. This was under Debian, I think it was working with both the gtk2 and Qt widget sets but have no immediate reason to think that it was a problem.

I'd have used TProcess for that, I suggest reading up on that and/or experimenting, then if you need more help I'll try to dig out my code as an example.

MarkMLl
Title: Re: Piping Console Output into GUI Application
Post by: engkin on June 09, 2021, 01:51:25 pm
Why, what's wrong with the code posted above by macrov in the *first* reply? :-\
Title: Re: Piping Console Output into GUI Application
Post by: marcov on June 09, 2021, 02:41:57 pm
My solution will only trap Pascal output.

If he means gtk errors etc, I don't know.
Title: Re: Piping Console Output into GUI Application
Post by: MarkMLl on June 09, 2021, 02:57:20 pm
Quote from: marcov on June 09, 2021, 02:41:57 pm
My solution will only trap Pascal output.

If he means gtk errors etc, I don't know.

Apologies if it looked like I was intentionally treading on your toes.

I've got working code that will run a shell, I think it also ran arbitrary programs without an intervening shell, and I put various things in there that allowed me to run a nested copy of Lazarus hence gdb to debug child programs... so at that point I'm fairly confident it will catch just about anything.

As a slight aside: Marco, am I correct in that the Lazarus IDE doesn't have an interactive shell-out facility?

MarkMLl
Title: Re: Piping Console Output into GUI Application
Post by: ThomasK on June 09, 2021, 04:23:16 pm
 
Quote from: marcov on June 09, 2021, 02:41:57 pm
My solution will only trap Pascal output.

If he means gtk errors etc, I don't know.

Maybe to make it more clear, if I start the application in a console "./libsockuse" or "sudo ./libsockuse" (this is at least required if I want to use it with root rights to set CAN configuration), it will show the (error) messages in that shell window. This is what I want to catch and show it in the application itself.

This is not necessarily Pascal output, as you can see in the screen shot.
Title: Re: Piping Console Output into GUI Application
Post by: Gustavo 'Gus' Carreno on June 09, 2021, 05:24:06 pm
Hey ThomasK,

I hope what I'm about to suggest make sense or is even correct/possible.

What Marco V. suggested is good to capture anything that comes out of the Pascal side the app. I'm not sure it will capture the GTK2 side of things...

But if one starts the app with both STDOUT and STDERR piped to a file, then we can use the timer technique that Marco V. suggested but directed at the file at the end of that pipe:
Code: Bash  [Select][+][-]
  1. $ ./MyGTK2App 1>&2 > output.txt

One then just has to check if the file exists and then update the TMemo with it's contents.

All the time crossing our fingers that the buffer for that file is flushed with enough regularity that something shows before the closing of the app.

OR I'm completely misunderstanding the way pipes work and I just should shut up and crawl back under my cozy little rock ;)

Cheers,
Gus
Title: Re: Piping Console Output into GUI Application
Post by: MarkMLl on June 09, 2021, 05:25:55 pm
Quote from: ThomasK on June 09, 2021, 04:23:16 pm
Maybe to make it more clear, if I start the application in a console "./libsockuse" or "sudo ./libsockuse" (this is at least required if I want to use it with root rights to set CAN configuration), it will show the (error) messages in that shell window. This is what I want to catch and show it in the application itself.

Right, so you're running from a shell: standard TProcess stuff IIRC. There's other possibilities e.g. using a pty, if you're stuck I'll look for examples later.

You don't need to use root, you can use POSIX capabilities although you might still need gdbserver for debugging.

MarkMLl
Title: Re: Piping Console Output into GUI Application
Post by: Gustavo 'Gus' Carreno on June 09, 2021, 05:37:29 pm
Hey MarkMLI,

I'm not sure the he wants the output of a third application.

If I've read him well, he wants to show the output of the application ITSELF, not of another one.

He mentions A debugger just for the fact that he can see the output of the debugged app in some window of the debugger.
I think that's where is interest on the debugger ends.

But again, I may be completely out of order and just muddying the waters a bit more. If so, I deeply apologise!!

Cheers,
Gus
Title: Re: Piping Console Output into GUI Application
Post by: ThomasK on June 09, 2021, 06:14:18 pm
Hi Mark,

Quote from: MarkMLl on June 09, 2021, 05:25:55 pm
Right, so you're running from a shell: standard TProcess stuff IIRC. There's other possibilities e.g. using a pty, if you're stuck I'll look for examples later.

No I am not running from a shell, this was just for the purpose to test all possible calls in the application. I use TProcess shell calls as sudo for the functions requiring root rights, here I get the stdout &stderr.

This because these functions are only used for setting up baudrate, start & stop the interface.

Code: Pascal  [Select][+][-]
  1. begin
  2.     OutputLines := TStringList.Create;
  3.     AProcess    := TProcess.Create(nil);
  4.     AProcess.Executable := '/bin/sh';
  5.     AProcess.Parameters.Add('-c');
  6.     AProcess.Parameters.Add(cmds2exec);
  7.     AProcess.Options := AProcess.Options + [poWaitOnExit, poUsePipes];
  8.     AProcess.Execute;
  9.     OutputLines.LoadFromStream(Aprocess.Output);
  10.     if length(Outputlines.text)>0 then
  11.       FCANTest.mlog.lines.addstrings(Outputlines.text);
  12.     OutputLines.LoadFromStream(AProcess.Stderr);
  13.     if length(Outputlines.text)>0 then
  14.       FCANTest.mlog.lines.addstrings(Outputlines.text);
  15.     AProcess.Free;
  16.     OutputLines.Free;
  17. end;  
  18.  
As also decribed in https://forum.lazarus.freepascal.org/index.php/topic,54783.msg408024.html#msg4080 (https://forum.lazarus.freepascal.org/index.php/topic,54783.msg408024.html#msg4080)

As Gus mentions, I want the outputs of processes which show up in the console if I would start from the console.

They are also shown to me in the Debug window if I start the application in Lazarus with debugging. (screenshot in the initial post)

Maybe due to the fact, that Lazarus is the caller of the application it can catch the console output and show it in the debug console input / output window.

thx.

Thomas


 
Title: Re: Piping Console Output into GUI Application
Post by: avra on June 09, 2021, 06:28:27 pm
Quote from: ThomasK on June 09, 2021, 06:14:18 pm
I want the outputs of processes which show up in the console if I would start from the console.
Take a look at RunProcess() at https://bitbucket.org/avra/ct2laz/src/master/utils.pas. It runs external process and captures it's output to a memo on both linux and windows.
Title: Re: Piping Console Output into GUI Application
Post by: marcov on June 09, 2021, 10:50:39 pm
Quote from: MarkMLl on June 09, 2021, 02:57:20 pm
As a slight aside: Marco, am I correct in that the Lazarus IDE doesn't have an interactive shell-out facility?

Afaik indeed it doesn't. It was designed for systems that could run more tasks in parallel, so there was no need for functionality to suspend the IDE to run a shell like old DOS IDEs did.
Title: Re: Piping Console Output into GUI Application
Post by: marcov on June 09, 2021, 10:54:20 pm
Quote from: ThomasK on June 09, 2021, 04:23:16 pm
Quote from: marcov on June 09, 2021, 02:41:57 pm
My solution will only trap Pascal output.

If he means gtk errors etc, I don't know.

Maybe to make it more clear, if I start the application in a console "./libsockuse" or "sudo ./libsockuse" (this is at least required if I want to use it with root rights to set CAN configuration), it will show the (error) messages in that shell window. This is what I want to catch and show it in the application itself.

Those might not even be application errors, but kernel (module) errors redirected to the console.

Maybe directly interacting with sysconsole pty or so?

Anyway I suggest to rethink that. It sounds simple, but it probably isn't. (and by that I mean that it might be hard to do that from _any_ process, not just a FPC compiled one)
Title: Re: Piping Console Output into GUI Application
Post by: Fred vS on June 09, 2021, 11:18:23 pm
Hello.

Thanks to Warfley (sorry I did not re-find the post), this will redirect all message-console, including gtk errors etc, into a text file.

Code: Pascal  [Select][+][-]
  1. program errorlog;
  2.   uses
  3.    ... ,  Classes, BaseUnix;
  4.    var
  5.      fs: TFileStream;
  6.    begin
  7.      fs := TFileStream.Create('error.log', fmOpenReadWrite or fmCreate);
  8.      FpDup2(fs.Handle, StdErrorHandle);
  9.        ...  // all your program stuff
  10.      Application.run; // Inside application, catch fs data
  11.      fs.free;
  12.     end.

Maybe you could catch the 'error.log' data in the program himself (close fs, then read error.log).
Title: Re: Piping Console Output into GUI Application
Post by: engkin on June 10, 2021, 02:34:38 am
Yes, I agree with Fred. I found a SO post (https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread) that solved this problem using Dup2 and pipes. This way you don't need a "file". I tested the idea on an Android terminal. Seemed to work:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   Classes, SysUtils, BaseUnix
  7.   {$IFDEF UNIX}
  8.   {$IFDEF UseCThreads}
  9.   ,cthreads
  10.   {$ENDIF}
  11.   {$ENDIF}
  12.   { you can add units after this };
  13. {
  14. https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread
  15. }
  16. function Test(AHandle:cInt; var APipe: tfildes):cint;
  17. var
  18.   saved: cint;
  19. begin
  20.   Result:=-1;
  21.   saved:=FpDup(AHandle);
  22.   if FpPipe(APipe)<>0 then
  23.   begin
  24.     WriteLn('FpPipe failed.');
  25.     exit;
  26.   end;
  27.   FpDup2(APipe[1], AHandle);
  28.   FpClose(APipe[1]);
  29.   Result:=saved;
  30. end;
  31.  
  32.  
  33. const
  34.   MAX_LEN=40;
  35. type
  36.   TBuffer=array[0..MAX_LEN-1] of char;
  37. var
  38.   Pipe: tfildes=(0,0);
  39.   saved_, TestHandle: cint;
  40.   buffer:TBuffer;
  41. begin
  42.   buffer:=Default(TBuffer);
  43.  
  44.   TestHandle:=StdOutputHandle;
  45.   //TestHandle:=Test(StdErrorHandle);
  46.  
  47.   WriteLn('Test Before');
  48.   saved_:=Test(TestHandle, Pipe);
  49.   WriteLn('Test After'); //<---- this will be sent to the pipe
  50.   Flush(StdOut);
  51.  
  52.   fpread(Pipe[0], buffer[0], MAX_LEN); // read from pipe into buffer
  53.  
  54.   FpDup2(saved_, StdOutputHandle);  // reconnect stdout for testing
  55.  
  56.   WriteLn(Format('read: %s', [buffer]));
  57.   readLn;
  58. end.

To test on Android:
Code: Text  [Select][+][-]
  1. adb push project1 "/data/local/tmp"
  2. adb shell chmod 755 /data/local/tmp/project1
  3. adb shell /data/local/tmp/project1

The output:
Quote
Test Before
read: Test After

On Linux, StdErrorHandle is hard coded:
Code: Pascal  [Select][+][-]
  1. const
  2.   UnusedHandle    = -1;
  3.   StdInputHandle  = 0;
  4.   StdOutputHandle = 1;
  5.   StdErrorHandle  = 2;

I believe it should work with kernel errors as well.
Title: Re: Piping Console Output into GUI Application
Post by: MarkMLl on June 10, 2021, 08:52:59 am
Quote from: marcov on June 09, 2021, 10:50:39 pm
Quote from: MarkMLl on June 09, 2021, 02:57:20 pm
As a slight aside: Marco, am I correct in that the Lazarus IDE doesn't have an interactive shell-out facility?

Afaik indeed it doesn't. It was designed for systems that could run more tasks in parallel, so there was no need for functionality to suspend the IDE to run a shell like old DOS IDEs did.

Thanks for that, just checking I've not overlooked anything. I've got a vague recollection of a couple of cases where it would have been useful although I can't remember the exact details... and of course there's the "shell-like" facilities of "change current directory" and "set shell/environment variable".

MarkMLl
Title: Re: Piping Console Output into GUI Application
Post by: Edson on June 10, 2021, 08:10:41 pm
Maybe I don't understand well the problem but in case you need to capture the output or error of a process , you can try my library: https://github.com/t-edson/UnTerminal

If you need to emulate a terminal, to control special inputs like "sudo" needs, I wrote an article about that: http://blogdetito.com/2016/12/04/el-inicio-de-un-terminal-con-linux-y-free-pascal/
Title: Re: Piping Console Output into GUI Application
Post by: ThomasK on June 25, 2021, 07:49:11 pm
Quote from: engkin on June 10, 2021, 02:34:38 am
Yes, I agree with Fred. I found a SO post (https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread) that solved this problem using Dup2 and pipes. This way you don't need a "file". I tested the idea on an Android terminal. Seemed to work:

I modified the code a little bit:
Code: Pascal  [Select][+][-]
  1. program test_pipe;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. {$Define std}
  6.  
  7. uses
  8.   Classes, SysUtils, BaseUnix
  9.   {$IFDEF UNIX}
  10.   {$IFDEF UseCThreads}
  11.   ,cthreads
  12.   {$ENDIF}
  13.   {$ENDIF}
  14.   { you can add units after this };
  15. {
  16. https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread
  17. }
  18. function Test(AHandle:cInt; var APipe: tfildes):cint;
  19. var
  20.   saved: cint;
  21. begin
  22.   Result:=-1;
  23.   saved:=FpDup(AHandle);
  24.   if FpPipe(APipe)<>0 then
  25.   begin
  26.     WriteLn('FpPipe failed.');
  27.     exit;
  28.   end;
  29.   FpDup2(APipe[1], AHandle);
  30.   FpClose(APipe[1]);
  31.   Result:=saved;
  32. end;
  33.  
  34.  
  35. const
  36.   MAX_LEN=40;
  37. type
  38.   TBuffer=array[0..MAX_LEN-1] of char;
  39. var
  40.   Pipe: tfildes=(0,0);
  41.   saved_, TestHandle: cint;
  42.   buffer:TBuffer;
  43. begin
  44.   buffer:=Default(TBuffer);
  45. {$ifdef std}
  46.   TestHandle:=StdOutputHandle;
  47. {$else}
  48.   TestHandle:=StdErrorHandle;
  49. {$endif}
  50.   WriteLn('Test Before');
  51.   saved_:=Test(TestHandle, Pipe);
  52.   WriteLn('Test After'); //<---- this will be sent to the pipe
  53. {$ifdef std}
  54.   Flush(StdOut);
  55. {$else}
  56.   Flush(StdErr);
  57. {$endif}
  58.  
  59.   fpread(Pipe[0], buffer[0], MAX_LEN); // read from pipe into buffer
  60. {$ifdef std}
  61.   FpDup2(saved_, StdOutputHandle);  // reconnect stdout for testing
  62. {$else}
  63.   FpDup2(saved_, StdErrorHandle);  // reconnect stderr for testing
  64. {$endif}
  65.  
  66.   WriteLn(Format('read from pipe: %s', [buffer]));
  67.   readLn;
  68. end.
  69.  

when the compiler switch is set for 'std' it works.

Code: Bash  [Select][+][-]
  1. pi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $ ./test_pipe
  2. Test Before
  3. read from pipe: Test After
  4.  
Finished as pogrammed with <Enter>

But when stderr should be used, the program stucks.
Code: Bash  [Select][+][-]
  1. pi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $ ./test_pipe
  2. Test Before
  3. Test After
  4.  
  5.  
  6.  
  7. ^C
  8. pi@RasPiHW:~/FreePascal/Projekte/Konsolenapps $
  9.  
It has to be terminated with Ctrl-c.

Any ideas?
Title: Re: Piping Console Output into GUI Application
Post by: engkin on June 25, 2021, 09:09:48 pm
It is waiting for you to hit enter (or somehow pass it if you use Android through ADB). The code for testing is more like:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   Classes, SysUtils, BaseUnix
  7.   {$IFDEF UNIX}
  8.   {$IFDEF UseCThreads}
  9.   ,cthreads
  10.   {$ENDIF}
  11.   {$ENDIF}
  12.   { you can add units after this };
  13. {
  14. https://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread
  15. }
  16. function Test(AHandle:cInt; var APipe: tfildes):cint;
  17. var
  18.   saved: cint;
  19. begin
  20.   Result:=-1;
  21.   saved:=FpDup(AHandle);
  22.   if FpPipe(APipe)<>0 then
  23.   begin
  24.     WriteLn('FpPipe failed.');
  25.     exit;
  26.   end;
  27.   FpDup2(APipe[1], AHandle);
  28.   FpClose(APipe[1]);
  29.   Result:=saved;
  30. end;
  31.  
  32. procedure Test_StdText(var f:Text);
  33. const
  34.   MAX_LEN=40;
  35. type
  36.   TBuffer=array[0..MAX_LEN-1] of char;
  37. var
  38.   Pipe: tfildes=(0,0);
  39.   saved_, TestHandle: cint;
  40.   buffer:TBuffer;
  41. begin
  42.   buffer:=Default(TBuffer);
  43.   TestHandle:=TextRec(f).Handle;
  44.  
  45.   WriteLn(f,'Test Before');
  46.   saved_:=Test(TestHandle, Pipe);
  47.   WriteLn(f,'Test After'); //<---- this will be sent to the pipe
  48.   Flush(f);
  49.  
  50.   fpread(Pipe[0], buffer[0], MAX_LEN); // read from pipe into buffer
  51.  
  52.   FpDup2(saved_, TextRec(f).Handle);  // reconnect stdout for testing
  53.  
  54.   WriteLn(Format('read: %s', [buffer]));
  55.   //readLn;
  56. end;
  57.  
  58. begin
  59.   WriteLn('StdOut:');
  60.   Test_StdText(StdOut);
  61.  
  62.   WriteLn('StdErr:');
  63.   Test_StdText(StdErr);
  64. end.

Pressing Ctrl+C is ending the local process.
TinyPortal © 2005-2018