Recent

Author Topic: Open Python with TProcess and send Input, get Output  (Read 6334 times)

penpen

  • New Member
  • *
  • Posts: 23
Open Python with TProcess and send Input, get Output
« on: December 16, 2018, 02:39:02 am »
Hello,
I'm trying to use Tprocess to read and write from/to python command line.

Sadly the command line returns nothing while python is running I only receive empty strings.

What I hope to do is open the python process and use it like I would from the command line.

Code: Pascal  [Select][+][-]
  1. var
  2. AProcess : Tprocess
  3. buf ..
  4. ...
  5. Begin
  6.     AProcess.SetCommandline('C:\...\python.exe');
  7.     AProcess.setOptions([poUsePipes]);            
  8.     AProcess.execute;
  9.   //Works fine till here
  10. // Then running this never returns anything
  11. repeat
  12. AProcess.Output().Read(buf, 1024);
  13. ...
  14. until ....;
  15.  
Hope you can help :)
                                     
« Last Edit: December 16, 2018, 02:40:52 am by penpen »

Renat.Su

  • Full Member
  • ***
  • Posts: 230
    • Renat.Su
Re: Open Python with TProcess and send Input, get Output
« Reply #1 on: December 16, 2018, 04:37:16 am »
Can You prodive some more code?
Example from code with commandline
Code: Pascal  [Select][+][-]
  1. var
  2.   CharBuffer: array [0..511] of char;
  3.   ReadCount: integer;
  4.   OutputString: String;
  5. const
  6.   SleepForOut = 100;  // Pause to wait for output
  7. begin
  8.   Sleep(SleepForOut);  
  9.  
  10.   OutputString:=EmptyStr;
  11.   while FProc.Output.NumBytesAvailable > 0 do
  12.   begin
  13.     ReadCount := FProc.Output.NumBytesAvailable;
  14.     if ReadCount>Length(CharBuffer) then
  15.       ReadCount:=Length(CharBuffer);
  16.     FProc.Output.Read(CharBuffer, ReadCount);
  17.     OutputString+=Copy(CharBuffer, 0, ReadCount);
  18. //    Write(StdOut, OutputString); // You can uncomment for debug
  19.   end;  
  20. ... ... ... ...
  21.  
  22.  

Renat.Su

  • Full Member
  • ***
  • Posts: 230
    • Renat.Su
Re: Open Python with TProcess and send Input, get Output
« Reply #2 on: December 16, 2018, 04:39:37 am »
You should check the output size before reading. In addition, you must wait (Sleep(SleepForOut);) for the result of the Python script to appear in the terminal
Sample code https://github.com/Al-Muhandis/ShellRemoteBot/blob/master/shellthread.pas
« Last Edit: December 16, 2018, 10:23:39 am by Renat.Su »

penpen

  • New Member
  • *
  • Posts: 23
Re: Open Python with TProcess and send Input, get Output
« Reply #3 on: December 16, 2018, 12:22:49 pm »
Thanks for the reply.
I tried putting some sleep time but it does not make a difference.

BTW with other processes the exact same code does work, but not with python. I tried almost everything.

Seems like I must miss something.

Anyone got it to work ? Or is python maybe not able to communicate through "pipes" ?

EDIT:
If I run a python script that exits, it does work and I get output. But it needs to exit before I can read the output. SO I'm still stuck  %)

Code: Pascal  [Select][+][-]
  1.  AProcess.SetCommandline('C:\...\python.exe scripty.py');
« Last Edit: December 16, 2018, 01:03:29 pm by penpen »

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Open Python with TProcess and send Input, get Output
« Reply #4 on: December 16, 2018, 01:12:19 pm »
I'm no Python expert, but maybe the python code buffers, and you need to insert flushes into the python code to force the python runtime to actually write the already generated output to disk.

P.s. setcommandline is deprecated. (and a private method?!?)

penpen

  • New Member
  • *
  • Posts: 23
Re: Open Python with TProcess and send Input, get Output
« Reply #5 on: December 16, 2018, 02:10:56 pm »
Thanks for the input.

The thing is I want to communicate with python both ways. Like.
Sending commands to python.
And then.
Receiving back the results.
Without restarting the python engine all the time.

I found a way to get the output but its pretty slow (around 200ms per command) when I use "TProcess.CloseInput;" I can read the output, but the problem is that it also exits python and I have to restart it with "TProcess.execute" then. Which is all rather slow.

This works but exits the python engine.
Code: Pascal  [Select][+][-]
  1.     var
  2.     AProcess : Tprocess
  3.     buf ..
  4.     ...
  5.     Begin
  6.         AProcess.SetCommandline('C:\...\python.exe');
  7.         AProcess.setOptions([poUsePipes]);            
  8.         AProcess.execute;
  9.        //We get output using this, but it closes python and It has to be restarted for more commands.
  10.     cmd:= 'print("testabc")'+#10;
  11.     Aprocess.getInput().Write(cmd[1],length(cmd));      
  12.     AProcess.CloseInput;
  13.       repeat
  14.         count := AProcess.Output.Read(buf, AProcess.Output.getNumBytesAvailable);
  15.       until AProcess.getOutput().getNumBytesAvailable = 0;  
  16.      


Edit:
This line takes like 200ms to process. Everything else is actually fast. Gotta test more later.
If anyone has an idea how to get it faster please share :) Would be much appreciated.

Code: Pascal  [Select][+][-]
  1. //very slow execution around 200ms
  2. AProcess.Output.Read(buf, AProcess.Output.getNumBytesAvailable);
« Last Edit: December 16, 2018, 03:06:21 pm by penpen »

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Open Python with TProcess and send Input, get Output
« Reply #6 on: December 16, 2018, 03:39:08 pm »
Try something like this: (or just call tprocess.readinputstream if you run 3.2 or 3.3.1)

Code: [Select]
Const
  READ_BYTES = 65536; // not too small to avoid fragmentation when reading large files.

function TProcessnamemacro.ReadInputStream(p:TInputPipeStream;var BytesRead:integer;var DataLength:integer;var data:string;MaxLoops:integer=10):boolean;
var Available, NumBytes: integer;
begin
    Available:=P.NumBytesAvailable;
    result:=Available>0;
    if not result then
     exit;
    while (available > 0) and (MaxLoops>0) do
      begin
        if (BytesRead + available > DataLength) then
          begin
            DataLength:=BytesRead + READ_BYTES;
            Setlength(Data,DataLength);
          end;
        NumBytes := p.Read(data[1+BytesRead], Available);
        if NumBytes > 0 then
          Inc(BytesRead, NumBytes);
        Available:=P.NumBytesAvailable;
        dec(MaxLoops);
      end;
end;

Note that it tries several times.

Edson

  • Hero Member
  • *****
  • Posts: 1301
Re: Open Python with TProcess and send Input, get Output
« Reply #7 on: December 16, 2018, 04:57:32 pm »
By default Python interpreter, works with a TTY or PTY console, not the stdin/stdout. But you can work using the -i option.
Using my library: https://github.com/t-edson/UnTerminal you can interact with Python, using events and prompt detection.
Lazarus 2.2.6 - FPC 3.2.2 - x86_64-win64 on Windows 10

penpen

  • New Member
  • *
  • Posts: 23
Re: Open Python with TProcess and send Input, get Output
« Reply #8 on: December 16, 2018, 06:48:49 pm »
By default Python interpreter, works with a TTY or PTY console, not the stdin/stdout. But you can work using the -i option.

That was it. Thank you very much :) Finally its working as expected.

Thanks everyone.

Renat.Su

  • Full Member
  • ***
  • Posts: 230
    • Renat.Su
Re: Open Python with TProcess and send Input, get Output
« Reply #9 on: December 16, 2018, 08:08:16 pm »
By default Python interpreter, works with a TTY or PTY console, not the stdin/stdout. But you can work using the -i option.
Using my library: https://github.com/t-edson/UnTerminal you can interact with Python, using events and prompt detection.
Cool repository!
It is a pity that there is no description in English. I wrote the simple Shell terminal emulator via telegram. Can you tell if there is any way to emulate sending Ctrl+C to the terminal? I use TProcess.Input and TProcess.Output

Edson

  • Hero Member
  • *****
  • Posts: 1301
Re: Open Python with TProcess and send Input, get Output
« Reply #10 on: December 17, 2018, 12:38:05 am »
It is a pity that there is no description in English.
Translations are welcome.  :)
Can you tell if there is any way to emulate sending Ctrl+C to the terminal? I use TProcess.Input and TProcess.Output
<Ctrl+C> in terminals is equivalent to the #3 character, so you just need to send that character.
Lazarus 2.2.6 - FPC 3.2.2 - x86_64-win64 on Windows 10

Renat.Su

  • Full Member
  • ***
  • Posts: 230
    • Renat.Su
Re: Open Python with TProcess and send Input, get Output
« Reply #11 on: December 17, 2018, 04:38:00 am »
<Ctrl+C> in terminals is equivalent to the #3 character, so you just need to send that character.
Thank You!!!

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Open Python with TProcess and send Input, get Output
« Reply #12 on: December 17, 2018, 05:36:25 am »
hello,
I'm trying to use Tprocess to read and write from/to python command line.             
instead of use TProcess you can use  the python for lazarus package
Example for this python script :
Code: Python  [Select][+][-]
  1. name = input("What's your name? \n")
  2. print("Nice to meet you " + name + "!")
  3. age = input("Your age? \n")
  4. print("So, you are already " + str(age) + " years old, " + name + "!")

in a form put a PythonEngine and a PythonGUIInputOutput components
code to run the python script :
Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. var
  3.   s: TStringList;
  4.  begin
  5.   s := TStringList.create;
  6.   try
  7.     s.LoadFromFile('M:\dev\python\pythontest\test_lazarus.py');
  8.     PythonEngine1.ExecStrings( s );
  9.   finally
  10.     s.free;
  11.   end;
  12. end;
Result  : see the Lazarus_python.gif file (click on the file to see animation)

Lazarus 1.8.2 32 bits on Windows 10 64 bits . Python 3.7.1 embedded 32 bits.

Friendly, J.P
« Last Edit: December 17, 2018, 05:39:18 am by Jurassic Pork »
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

 

TinyPortal © 2005-2018