Recent

Author Topic: Reading large process output not working  (Read 13219 times)

vaspervnp

  • Newbie
  • Posts: 5
Reading large process output not working
« on: July 28, 2013, 04:27:05 pm »
I am trying to make a GUI frontend for a command line program but I can't seem to get Lazarus to read from the process output. Even the code here http://wiki.lazarus.freepascal.org/Executing_External_Programs#Reading_large_output  isn't working. It gets stuck at 
Code: [Select]
NumBytes := OurProcess.Output.Read((MemStream.Memory + BytesRead)^, READ_BYTES);until I close the console window or terminate the process. I also can't seem to find anyone who has managed to create a gui for a running program. I have done this many times with Delphi, and all I can think of is that Lazarus just can't do it yet. Is this the case?

Leledumbo

  • Hero Member
  • *****
  • Posts: 8790
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Reading large process output not working
« Reply #1 on: July 28, 2013, 04:55:08 pm »
Quote
Even the code here http://wiki.lazarus.freepascal.org/Executing_External_Programs#Reading_large_output  isn't working. It gets stuck
Give us a minimalistic compilable program that demonstrate this behavior. I was a quite frequent user of TProcess with such needs.
Quote
I also can't seem to find anyone who has managed to create a gui for a running program
https://code.google.com/p/express-gui/
https://code.google.com/p/octave-gui/
I also have a terminal wrapper and some other console programs' wrapper.
Quote
I have done this many times with Delphi, and all I can think of is that Lazarus just can't do it yet. Is this the case?
Certainly NO, how do you think Lazarus controls the compiler?

vaspervnp

  • Newbie
  • Posts: 5
Re: Reading large process output not working
« Reply #2 on: July 28, 2013, 05:50:38 pm »
This example here doesn't work for me:
http://wiki.lazarus.freepascal.org/Executing_External_Programs#Reading_large_output

I am trying to use it with minerd, a console litecoin miner, so the only difference is that the wrapper is gui based, just a form with a start button and a memo for the output for now. This is the only difference with the example.

The only thing I can think off is that the example is written wrong and something is missing, but I have no idea what it might be.

I am going to look into the code for calling the compiler, but I thought someone could give me a quick answer before I go into thousands of lines of code.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8790
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Reading large process output not working
« Reply #3 on: July 28, 2013, 06:13:42 pm »
Quote
I am trying to use it with minerd, a console litecoin miner, so the only difference is that the wrapper is gui based, just a form with a start button and a memo for the output for now. This is the only difference with the example.
Surely that's not ONLY. The console program's behavior must be known in order to be controlled well. Does it buffer its output until the last second? Does it expect some input before terminating? etc, etc.
Quote
The only thing I can think off is that the example is written wrong and something is missing, but I have no idea what it might be.
The example works just fine (just tested here). You haven't shown any of your code, from which we don't know what's wrong.

vaspervnp

  • Newbie
  • Posts: 5
Re: Reading large process output not working
« Reply #4 on: July 28, 2013, 06:47:16 pm »
This is my program:
http://www.vasper.biz/files/minerdgui.rar

it just freezes on this line with no apparent reason:
Code: [Select]
NumBytes := OurProcess.Output.Read((MemStream.Memory + BytesRead)^, READ_BYTES);
I am certain the output is continuous because I have allready created a gui from Delphi for it.
« Last Edit: July 28, 2013, 06:59:44 pm by vaspervnp »

Mike.Cornflake

  • Hero Member
  • *****
  • Posts: 1263
Re: Reading large process output not working
« Reply #5 on: July 28, 2013, 08:29:01 pm »
Sorry, haven't downloaded your code (stuck with limited internet), so don't know if I'm going an tangent...

I had some problems with TProcess hanging on read.  In my case it was the program I was controlling was sending data out on the error console.  I resolved by adding poStderrToOutPut to TProcess.Options.
Code: [Select]
OurProcess.Options := [poNoConsole, poUsePipes, poStderrToOutPut] ;
Lazarus Trunk/FPC Trunk on Windows [7, 10]
  Have you tried searching this forum or the wiki?:   http://wiki.lazarus.freepascal.org/Alternative_Main_Page
  BOOKS! (Free and otherwise): http://wiki.lazarus.freepascal.org/Pascal_and_Lazarus_Books_and_Magazines

wp

  • Hero Member
  • *****
  • Posts: 12689
Re: Reading large process output not working
« Reply #6 on: July 28, 2013, 11:03:04 pm »
Quote
This is my program:
http://www.vasper.biz/files/minerdgui.rar

Be careful. 11 out of 46 virus scanners of www.virustotal.com report this file to be malicious.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8790
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Reading large process output not working
« Reply #7 on: July 28, 2013, 11:17:37 pm »
Quote
This is my program:
http://www.vasper.biz/files/minerdgui.rar
Please attach or link to either 32-bit windows exe/linux elf/plain source of the minerd executable, I don't have 64-bit system to test.
« Last Edit: July 28, 2013, 11:24:43 pm by Leledumbo »

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1255
Re: Reading large process output not working
« Reply #8 on: July 29, 2013, 01:14:22 am »
hello vaspervnp,
Quote
The only thing I can think off is that the example is written wrong and something is missing, but I have no idea what it might be.

1 - you have changed the code, so don't say that it is wrong  :-X !

2 - this code use text console output, so in the options of your project don't check the option Application Win32 Gui ( checked you cannot use the write writeln of the code). 

for your consoleExecute code  try this :
Code: [Select]
interface
uses
  Classes, Process, SysUtils, forms, Controls, Graphics, StdCtrls,LConvEncoding;

procedure getConsoleOutput(command:string;memo:TMemo);

implementation

procedure getConsoleOutput(command:string;memo:TMemo);
const
  READ_BYTES = 2048;

var
  OurCommand: String;
  OutputLines: TStringList;
  MemStream: TMemoryStream;
  OurProcess: TProcess;
  NumBytes: LongInt;
  BytesRead: LongInt;
  i:Integer;
begin
  // A temp Memorystream is used to buffer the output
  MemStream := TMemoryStream.Create;
  BytesRead := 0;

  OurProcess := TProcess.Create(nil);
  // Recursive dir is a good example.
  OurCommand:='invalid command, please fix the IFDEFS.';
  {$IFDEF Windows}
  //Can't use dir directly, it's built in
  //so we just use the shell:
  OurCommand:=command;
  {$ENDIF Windows}
  {$IFDEF Unix}
  //Needs to be tested on Linux/Unix:
  OurCommand := command;
  {$ENDIF Unix}
  OurProcess.CommandLine := OurCommand;

  // We cannot use poWaitOnExit here since we don't
  // know the size of the output. On Linux the size of the
  // output pipe is 2 kB; if the output data is more, we
  // need to read the data. This isn't possible since we are
  // waiting. So we get a deadlock here if we use poWaitOnExit.
  OurProcess.Options := [poUsePipes];
  WriteLn('-- External program run started');
  OutputLines := TStringList.Create;
//  OurProcess.ShowWindow := swoHide;
  OurProcess.Execute;
  while OurProcess.Running do
  begin
    // make sure we have room
    MemStream.SetSize(BytesRead + READ_BYTES);

    // try reading it
    NumBytes := OurProcess.Output.Read((MemStream.Memory + BytesRead)^, READ_BYTES);
    if NumBytes > 0
    then begin
      Inc(BytesRead, NumBytes);
      Write('.') //Output progress to screen.
      //   here is your fatal error code
      //      OutputLines.LoadFromStream(MemStream);
      //for i := 0 to OutputLines.Count - 1 do
      //begin
      //  memo.lines.add(OutputLines[i]);
      //end;
    end
    else begin
      // no data, wait 100 ms
      Sleep(100);
    end;
  end;
  // read last part
  repeat
    // make sure we have room
    MemStream.SetSize(BytesRead + READ_BYTES);
    // try reading it
    NumBytes := OurProcess.Output.Read((MemStream.Memory + BytesRead)^, READ_BYTES);
    if NumBytes > 0
    then begin
      Inc(BytesRead, NumBytes);
      Write('.');
    end;
  until NumBytes <= 0;
  if BytesRead > 0 then WriteLn;
  MemStream.SetSize(BytesRead);
  WriteLn('-- External program run complete');


  OutputLines.LoadFromStream(MemStream);
  WriteLn('-- External program output line count = ', OutputLines.Count, ' --');
  for NumBytes := 0 to OutputLines.Count - 1 do
  begin
  // if you are in latin western europe country like me, convert OEM strings
  //  memo.lines.add( CP850ToUTF8(OutputLines[NumBytes]));
    memo.lines.add(OutputLines[NumBytes]);
//    WriteLn(OutputLines[NumBytes]);
  end;
  WriteLn('-- Program end');
  OutputLines.Free;
  OurProcess.Free;
  MemStream.Free;
end;

with this code, the memo is written only when the process is terminated. You can not see the output while the process is running.

Friendly, J.P
« Last Edit: July 29, 2013, 01:37:48 am by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

vaspervnp

  • Newbie
  • Posts: 5
Re: Reading large process output not working
« Reply #9 on: July 29, 2013, 08:47:07 am »
hello vaspervnp,
Quote
The only thing I can think off is that the example is written wrong and something is missing, but I have no idea what it might be.

1 - you have changed the code, so don't say that it is wrong  :-X !

2 - this code use text console output, so in the options of your project don't check the option Application Win32 Gui ( checked you cannot use the write writeln of the code). 


with this code, the memo is written only when the process is terminated. You can not see the output while the process is running.

Friendly, J.P

Thank you for the code, but unfortunately I need to get the output during execution, not at the end.

The changes I have made are small, so I don't see how they effect the output.

It might be the GUI setting, so I am going to try that and post my findings. Thanks again.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8790
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Reading large process output not working
« Reply #10 on: July 29, 2013, 04:06:08 pm »
One thing you could try is to test the example as-is without changing anything and see if it works on your system. If it works, then it's indeed your way of handling the console program that is wrong, not the example.

vaspervnp

  • Newbie
  • Posts: 5
Re: Reading large process output not working
« Reply #11 on: July 29, 2013, 04:54:23 pm »
Sure, but even if the command line works, I don't need a command line. I need a gui. I also don't need writeln, I need to get the data in a string. The program gets stuck on a line that hasn't changed and is before the code that writes anywhere. So, either the minerd is a strange console application, or there is something else I am missing... Thanks anyways.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8790
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Reading large process output not working
« Reply #12 on: July 29, 2013, 05:10:16 pm »
Quote
Sure, but even if the command line works, I don't need a command line. I need a gui. I also don't need writeln, I need to get the data in a string
The example doesn't differentiate console or gui app as host, the console application's output is buffered in a TMemoryStream, which you can use for anything (output to memo, write to your own console, dump to file, whatever).

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1255
Re: Reading large process output not working
« Reply #13 on: July 30, 2013, 12:38:08 am »
look at   :o  this topic :
http://forum.lazarus.freepascal.org/index.php/topic,17315.msg95224.html#msg95224

shellcommandrunner seems to be what you are looking for
« Last Edit: July 30, 2013, 10:04:02 am by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

russellman

  • Newbie
  • Posts: 3
Re: Reading large process output not working
« Reply #14 on: November 09, 2023, 09:11:41 pm »

https://drive.google.com/file/d/1-ajoqVmw2-WeLMG5f2CR33__fjK6WZY8/view?usp=sharing

{
 Gary Russell (ISSW)
 ghrussell@yahoo.com
 Create a console window for input.
 If you close the console window.
 The GUI will close too.
 Except on Program Console button.

 Show 3 way to use the console.

 Buttons:
   Open Console button.
     Open console for enput.
     Press the Q hey end input.
   Memo console button.
     Open console for input.
     Press the Q hey end input.
     But send the output to Memo1.
   Program Console button.
     Open a console.
     Uses RunCommand to do a dir command.
     dir /s is all files in the directory and it's sub-directorys.
     And put the Console output into a string s.
     Close the console.
     Copy the s string into the memo box.
     Note:
     If you use Program Console on large directory/sub-directory.
     It will hold the console window open for a long time.
}

unit Unit1;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, StdCtrls,
  Windows, Process;

type

  { TForm1 }

  TForm1 = class(TForm)
    Button1: TButton;
    Button2: TButton;
    Button3: TButton;
    Button4: TButton;
    Memo1: TMemo;
    SelectDirectoryDialog1: TSelectDirectoryDialog;
    procedure Button1Click(Sender: TObject);
    procedure Button2Click(Sender: TObject);
    procedure Button3Click(Sender: TObject);
    procedure Button4Click(Sender: TObject);
  private

  public

  end;

const
  BUF_SIZE = 2048; // Buffer size for reading the output in chunks

var
  Form1: TForm1;
  FileN: String;

implementation

{$R *.lfm}

{ TForm1 }

procedure TForm1.Button1Click(Sender: TObject);
var
  s: string;
begin
  ///AllocConsole initializes standard input, standard output,
  // and standard error handles for the new console.
  AllocConsole;      //in Windows unit. Win32 API.
  ///True for console applications, False for GUI applications.
  IsConsole := True; //in System unit.
  ///Initialization of standard file variables
  // (Input, Output, ErrOutput, StdOut and StdErr).
  SysInitStdIO;      //in System unit.
  ///Write to the console window.
  writeln('Enter something or Q to Quit');

  ///Repeat ontell you type in a capital Q.
  repeat begin
    ///Read Line. Read a line of text from the keyboard.
    // (Enter ends the line) And put the line into a string s.
    readln(s);
    ///After you press the Enter key.
    // Print 'You typed: ' and the string s.
    // To the console.
    writeln('You typed: ' + s);
  ///When you enter Q and press the Enter key. Loop ends.
  end until uppercase(s)='Q';

end;
///Memo console button.
procedure TForm1.Button2Click(Sender: TObject);
var
  s: string;
  ///A string List.
  StrList: TStringList;
begin
  AllocConsole;      // in Windows unit
  IsConsole := True; // in System unit
  SysInitStdIO;      // in System unit
  writeln('Enter something or Q to Quit');
  ///Allocate memory for the String List.
  StrList := TStringList.Create;

  repeat begin
    readln(s);
    ///Add the String s to the String List.
     StrList.Add(s);

    writeln('You typed: ' + s);

  end until uppercase(s)='Q';

  ///Output the StringList to Memo1.
  Memo1.Lines.Text := StrList.Text;
  ///Deallocate memory used by the StringList.
  StrList.Free;

end;

///Program Console button.
procedure TForm1.Button3Click(Sender: TObject);
var
  ///Create dynamic string.
  s: string;
begin

  ///Salect a directory and put it into FileN
  if SelectDirectoryDialog1.Execute then
    FileN := SelectDirectoryDialog1.FileName;

  ///Execute a command in the current working directory.
  // And put the output in string s.
  // dir /s is all files in the directory and it's sub-directorys.
  RunCommand('c:\windows\system32\cmd.exe', ['/c', 'dir /s ' + FileN], s);

  ///Put the string s into memo1.
  // Memo1.Lines is a AnsiString.
  // Memo1.Lines.Text is a string.
  Memo1.Lines.Text := s;
end;
///Exit
procedure TForm1.Button4Click(Sender: TObject);
begin
  Form1.Close;
end;

end.




 

TinyPortal © 2005-2018