[SOLVED] Bounty: Automate external process with prompts

[macmike]

Try my damn old OctaveGUI. I didn't wrap any shell, just execute the required process directly. I don't really remember the code, probably there was Windows specific code/treatment, but you can start with that.

Thanks that looks really interesting, I think you might have been lucky in the way Octave works but I'll look into it more deeply

[macmike]

https://github.com/alrieckert/lazarus/blob/master/debugger/pseudoterminaldlg.pp

the mentioned TPseudoTerminalDlg uses debugwindows, (c.f. my above messages = keyboard hooking = implies to install a debugger , also implies hard time to make it totally cross-platform)

believe me , lol , threads safe OSes are built to BE threads safe and Process safe, if you "inject data" by a back door it is

1) considered Hacking
2) not standard
3) unportable
4) breaking thru the security and making happy holes and open doors to pirats

I disagree, there's no thread safety issues here. As for process safety, inter-process communication is a standard feature of OSs, the challenge is doing it in a nice cross platform way using Lazarus. I'm not suggesting using a back door to "inject data" but using a published command line api. I can do what I'm trying to achieve in other technologies (how do you think terminal emulators are written?) - what I'm interested in is using Lazarus to achieve the same result.

[sam707]

you said "interaction" = getting answers from , then put orders to = injecting , call it as you want , blue cow if you like , its the same

IPC is the linux version of COM/Ole , its based on binary messages , I know right , I use it TY

[sam707]

Furthermore :

As I am an Emulator Fan, I am on my way to make what you call a Terminal emulator 

AND

so far mixing TCmdBox , TAsyncProcess plus a OEM to UTF8 translation , I really DO NOT encounter any problem.

2nd day of work (let say 10h) and I am here as follow :

1) I built some embeded commands begining with '!!'

like !!help , !!ver, !!cls, !!clh (clear history), !!bye, !!xitc (get exit code from last task) etc etc

2) I can launch any program or any embeded OS command with their parameters from inside My box by typing exactly the same way I would in a terminal

3) any launched task displays exactly what it would show inside a normal terminal, and I can catch the Exit Code

4) when the running task takes longer than 50 milisecond , 2 disabled buttons on the right of My emulated terminal become enabled with these functions : "Pause/Resume" , "Stop" the current task. these buttons return to grayed while no task active.

5) finally I plan to embed either PascalScript engine in the background, either my own simplified using commands like !!if , !!exist, !!fileage, !!echo(on/off), etc

COOL 

Code: [Select]

...
type
  TMyAsyncProcess = class(TAsyncProcess)
  ...
  end;
  { TMainForm }

  TMainForm = class(TForm)
    cpEdit: TEdit;
    Label2: TLabel;
    Label3: TLabel;
    StopButton: TButton;
    CmdBox: TCmdBox;
    Label1: TLabel;
    PauseTBox: TToggleBox;
    procedure CmdBoxOnEnter(Sender: TObject);
    procedure CmdBoxOnInput(ACmdBox: TCmdBox; Input: string);
    procedure FormCloseQuery(Sender: TObject; var CanClose: boolean);
    procedure FormCreate(Sender: TObject);
  private
    { private declarations }
    rdbuf: array[0..8191] of Byte;
  protected
    CanExit: boolean;
    console: TMyAsyncProcess;
    procedure WriteHelp;
    procedure ConsoleRead(Sender: TObject);
    procedure ConsoleExec;
    procedure cpChange;
   ..............
procedure TMainForm.CmdBoxOnInput(ACmdBox: TCmdBox; Input: string);
...
{$IF DEFINED(WINDOWS)}
  console.Executable:='cmd.exe';  // My AsyncProcessConsole ;)
  console.Parameters.Add('/C');
{$ELSEIF DEFINED(LINUX)}
  console.Executable:='/bin/sh';
  console.Parameters.Add('-i');
{$ELSE}
{$ERROR this must be compiled under windows or linux}
{$ENDIF}
  console.CommandLine:=console.Executable+' '+console.Parameters[0]+' '+Input;
......

... to be continued

[macmike]

Yep, that's not dissimilar to my example previously. The problem is that technique won't pick up prompts and can't respond to them.

I think that what the IDE does (on linux/mac) is use GDB to reset the current processes tty console. So perhaps a way to do this is to do the same thing, or just redirect /dev/tty on linux/mac?

[sam707]

Yep, that's not dissimilar to my example previously. The problem is that technique won't pick up prompts and can't respond to them.

I think that what the IDE does (on linux/mac) is use GDB to reset the current processes tty console. So perhaps a way to do this is to do the same thing, or just redirect /dev/tty on linux/mac?

sorry hehehe, I was wrong, I found out the way to respond to prompts with pipes. I love bounties roflmao

[sam707]

try this , read the readme.txt file 1st

It works fine ! on windows at least

[sam707]

patch to mainunit , fixing conditional string conversion upon empty codepage :

Code: [Select]

procedure TMainForm.InjectButtonClick(Sender: TObject);
var
  r: integer;
  s: string;
  p: PChar;
  len: longint;
begin
  AnsForm.Edit.Text:=EmptyStr;
  r:=AnsForm.ShowModal;
  if r<>mrOk then Exit;
  try
    if cpEdit.Text<>EmptyStr then
      s:=ConvertEncoding(AnsForm.Edit.Text,GetDefaultTextEncoding,cpEdit.Text)
    else
      s:=AnsForm.Edit.Text;
  Except
    On Exception do s:=AnsForm.Edit.Text;
  end;
  s:=s+#$0D; p:=PChar(s);
  len:=strlen(p);
  Move(p^,wrbuf,Succ(len));
  CmdBox.TextColor(clAqua); CmdBox.Writeln(AnsForm.Edit.Text);
  console.Input.Write(wrbuf,len);
end;

and

Code: [Select]

procedure TMainForm.ConsoleRead(Sender: TObject);
var
  nb,rd: dword;
  p: PChar;
  s: string;
begin
  nb:=console.NumBytesAvailable;
  rd:=console.Output.Read(rdbuf,nb);
  rdbuf[rd]:=0;
  p:=@rdbuf;
  try
  if cpEdit.Text<>EmptyStr then
    s:=ConvertEncoding(PChar(@rdbuf),cpEdit.Text,EncodingUTF8)
  else
    s:=PChar(@rdbuf);
  Except
    On Exception do s:=PChar(@rdbuf);
  end;
  CmdBox.Write(s);
end;


[sam707]

why do we need a script system ?

simply because we want "automated" answers to prompts instead of giving them manually, using the "Answer" button.

So

with a script command like !!onscan "text","answer" <console script launch command> , my T2 could analyse output text from console scripts and react properly automatically by injecting the proper answer.

even more !!onscan2 "datafile" <console script lauch command> could scan question patterns from the console output and inject the proper answerS
with a "datafile" looking like

enter your name, John
enter your age, 33
want to rebuild or compile, C
...

that sounds easy to do : before the comma, the piped output sentence to scan and check, after the comma, the answer to write to the piped input

[macmike]

Thanks Sam, I think you're getting a bit carried away with writing a terminal app, for my requirements at least. I'd love to see a Lazarus terminal app but that's not what my bounty is for. To clarifiy I need a solution (preferably just a class) that:

- allows invocation of an arbitrary command line application
- catches all textual output (stdout and stderr)
- catches or somehow detects applications raising prompts (this is killer requirement #1, think sudo or something to test it)
- allows sending a response to prompts and continuing execution (this is killer requirement #2)
- doesn't create a visible cli/xterm console window
- works on linux, mac and windows platforms (32/64)

I don't really want any other dependencies such as pascalscript, cmdbox etc. I just want a class that can do this stuff. Simple putting a memo and a bunch of buttons on form is enough for me. If someone adds all that stuff it's just work for me to strip it out later.

For those interested on working this that are wary of the Mac requirement, don't worry too much, in my experience solutions that work on linux tend to work on Macs. The bounty would still be up for grabs at 80% for a solution that's just windows and linux.

[macmike]

Here's the interface for my ideal solution (details aren't important):

Code: [Select]

  TCmdLineAutomation = class(TObject)
  public
    property CurrentDirectoy : string read FCurrentDirectoy write SetCurrentDirectoy;
    property IsRunning : boolean read FIsRunning write SetIsRunning;
    property CommandLine : string read FCommandLine write SetCommandLine; //or cmd + args

    //input
    property OnPrompt : TNotifyEvent read FOnPrompt write SetOnPrompt;
    property LastPrompt : string read FLastPrompt write SetLastPrompt;
    procedure SendInput(const inputStr : String);

    //output
    property OnOutputRecieved : TNotifyEvent read FOnOutputRecieved write SetOnOutputRecieved;
    property Output : TOutputStream read FOutput write SetOutput;
  end;     


[macmike]

Yeah, probably needs an execute command too 

[macmike]

This bounty is still open if anyone is interested.

[macmike]

This bounty is now closed.

The experience of running a bounty was a weird one. There were a few time wasters along the way and then finally two serious attempts to win it - in the same two weeks after 2 years of being open!

I created some tests: https://github.com/macmike/console_automation_tests/releases/tag/v0.1
Which were all passed today: https://github.com/macmike/console_automation_tests/releases/tag/v0.2

Thanks to skalogryz