capture apl_exec stdout

[Thaddy]

CIN COUT and CERR already map to stdin, stdout and stderr. Different names, but exactly the same.

[Thaddy]

of interest : for full display when initializing libapl i use
init_libapl(paramstr(0), 1);

Then you can change 0 to 1 if that is preferred, e.g.:

Code: Pascal  [Select][+][-]

1. procedure init_libapl(progname : string; log_startup : integer = 1);inline;
2. // and subsequently
3. init_libapl(Paramstr(0));

Since that a.o. means that libapl inherits the process handle (Paramstr(0)) that actually means that redirecting the output is much easier than I initially thought.

[TRon]

Can you give us the modifications? By the end of the week we have a package  !

No problem. What I have is attached.

I hope you did not touch the wrappers, though: that is pure pascalification and has almost no speed penalty because I inlined it.

https://www.youtube.com/watch?v=otCpCn0l4Wo 

If you want to capture all the output to terminal in Unix OS, you may use FpDup2

Yes the last resort is to try redirect all terminal output.

Your example works as expected (thank you very much for that example code Fred vS).

The only drawback is the need for a file (or files in this case: err and out)

example 3 in the attachment show the use of your approach.

i mistakingly said compiling --with-android made no difference (i forgot to copy the new libapl to /usr/local/lib/apl/libapl.so)  before compiling

No problem. I was expecting erors like that would surface when compiling for android.

Thanks you for the feedback toby.

would setting COUT to stdout be this solution for the stdout problem in the fpc code using libapl?

No, unfortunately that would not solve the issue. Its pretty technical, especially on/for Linux. E.g. I am trying to find a better solution but I am also not very well versed in such matters.

More important (e.g. first step) is that the library works as expected 

attachment update 2: I have re-arranged the examples, included some original documentation from APL trunk and added the two callback related functionality (with very simple invocation examples)
attachment update 2b: Added Thaddy's additions.
attachment update 3: almost all API calls are now available. Added, extended callback example, redirection example, license (PIF) and ulibapl.pas can now be compiled in mode Delphi as well.

[Thaddy]

Thanks for the code.
Btw, I also had strDispose(), but that is really not necessary because the temp PChar is allocated on the stack and so disappears after the function returns (goes out of scope). StrDispose is to free heap memory, see the documentation for strDispose, temp is stack.

I think I also solved the IO problem, stdout redirection works:

Code: Pascal  [Select][+][-]

1. {$mode objfpc}{$H+}
2. uses classes,streamIO, libaplu;
3. var
4.   OldOutput,
5.   f: TextFile;
6.   s: TStringStream;
7. begin
8.   s := TStringStream.Create;
9.   AssignStream(f, s);
10.   Rewrite(f);
11.   OldOutput:=Output;
12.   Output:=f;
13.   init_libapl(paramstr(0),1);
14.   { any output from apl should  be redirected }
15.   Writeln(Boolean(apl_command('2 < 1')));
16.   Close(f);
17.   Output:=OldOutPut;
18.   writeln('redirected: ', s.datastring);
19.   s.Free;
20. end.

AFAICT APL uses the same handles as the pascal code.
Needs some more work for stdin and stderr.
Please test. If it does not work I will send you a patched libapl. (very crude, but also works)

[toby]

Thaddy

are you using the same libaplu.pas that you posted earlier?

i had to add {$linklib apl} to top before uses ? don't you need it?  what is your compile line or /etc/fpc.cfg change?

and get
stdout.pas(18,15) Hint: Variable "f" does not seem to be initialized
stdout.pas(31,9) Error: Illegal type conversion: "AnsiString" to "Boolean"
stdout.pas(44) Fatal: There were 1 errors compiling module, stopping

if i add f := stdout;  right after begin
i now get
stdout.pas(31,9) Error: Illegal type conversion: "AnsiString" to "Boolean"

if i change
//Writeln(Boolean(apl_command('2 < 1')));      the '2 < 1' is an apl expression not an apl command
Writeln(Boolean(apl_exec('2 < 1')));

i get
0
redirected: FALSE

the result of  '2 < 1' is 0 not false
the false comes from the stdout of '0' and then your boolean(0) that is the stdout from all apl_exec expressions signifying sucdess with current libapl

use '2 < 1 1 1' to clarify if you are really getting apl_exec stdout and eliminate '0' 'false' confusion

---

using writeln(apl_command(')help'));         does indeed work
writeln('length(s.datastring) : ', length(s.datastring));
4506

---

[Thaddy]

Yes, there were some differences between the code I posted and the code I was working with. Correct that tommorow.
But did the redirect work for you?

[toby]

I have modified my post 19 since you did post 20

using apl_command(')help'); is redirected
but apl_exec('2 < 1 1 1');   doesn't

[marcov]

If you use commandline style redirection, either the library function you use needs to execute a shell (like system())) , or you have to specify a shell yourself.

[TRon]

Btw, I also had strDispose(), but that is really not necessary because the temp PChar is allocated on the stack and so disappears after the function returns (goes out of scope). StrDispose is to free heap memory, see the documentation for strDispose, temp is stack.

Sorry for that.

Please test. If it does not work I will send you a patched libapl. (very crude, but also works)

It works. That is the Output is redirected as expected. Thank you for that as I was not aware that you could 'trick' IO that way.

Unfortunately doing the same with ErrOutput does not. I assume the library uses the special UERR handle for that. (I assume(d) you have figured that yourself )

If you use commandline style redirection, either the library function you use needs to execute a shell (like system())) , or you have to specify a shell yourself.

Skimming posts can be risky, as is in this case 

Your input is appreciated but the used characters have nothing to do with redirection. It is how the APL language itself operates.

TS is trying to retrieve the output from the library (that is emitted to the terminal). I assume he wishes to do that so that the library can be used in a Lazarus GUI program.

[Fred vS]

If you want to capture all the output to terminal in Unix OS, you may use FpDup2

Yes the last resort is to try redirect all terminal output.

Your example works as expected (thank you very much for that example code Fred vS).

The only drawback is the need for a file (or files in this case: err and out)

Hello.

Yes, indeed it needs a file but only one.
If you want to have err and out in one file, in your apl_test3.pas, just use one TFileStream and 2 fpdup2():

Code: Pascal  [Select][+][-]

1. ...
2.  
3.   fs1 := TFileStream.Create(prglog1, fmOpenReadWrite or fmCreate);
4.  
5.   fpdup2(fs1.handle, StdErrorHandle);
6.   fpdup2(fs1.handle, StdOutputHandle);
7. ...
8.  

Maybe there is a trick to use a TMemorysteam vs a TFileStream for fpdup2 but I did not find the way.

[Thaddy]

Maybe there is a trick to use a TMemorysteam vs a TFileStream for fpdup2 but I did not find the way.

What you need is a THandleStream or descendant (that may behave like a memorystream).
That's because fpdup2 needs an OS handle.
The unit process contains TInputPipeStream and TOutputPipeStream, which descent from THandlestream.
These work with fpdup2.

Note that my approach is "a bit"  more cross-platform than fpdup2 and works under linux and windows, probably every platform that accepts objfpc. fpdup2 is unix only.
The two streams mentioned are also X-platform and indepent of the TProcess class.

[Thaddy]

I have added the *_ucs calls and overloaded the pascal wrappers for UCS/Unicode:

Code: Pascal  [Select][+][-]

1. unit libaplu;
2. {$mode delphi}{$H+ string=ansistring}
3. interface
4. { Pascal function wrappers }
5. procedure init_libapl(progname : string; log_startup : integer = 1 {based on Toby's suggestion });inline;
6. function apl_exec(line_utf8 : string) : integer;inline;overload;
7. function apl_exec(line_utf8 : unicodestring) : integer;inline;overload;
8. function apl_command(command_utf8 : string):string;inline;overload;
9. function apl_command(command_utf8 : unicodestring):unicodestring;inline;overload;
10.  
11. { external declarations, better use the pascal wrappers }
12. procedure init_libapl(progname : PChar; log_startup : longint);cdecl;external;
13. function apl_exec(line_utf8 : pchar) : longint; cdecl; external;
14. function apl_exec_ucs(line_utf8 : pWidechar) : longint; cdecl; external;
15. { note if you use the next functions with a global variable for the result,
16.   you must call dispose. For local variables this is not needed. }
17. function apl_command(command_utf8 : pchar) : pchar; cdecl; external;
18. function apl_command_ucs(command_utf8 : pWidechar) : pWidechar; cdecl; external;
19.  
20. implementation
21.  
22. procedure init_libapl(progname : string; log_startup : integer);inline;
23. begin
24.   init_libapl(Pchar(progname),log_startup);
25. end;
26.  
27. function apl_exec(line_utf8 : string) : integer;
28. begin
29.    Result :=apl_exec(PChar(line_utf8));
30. end;
31.  
32. function apl_command(command_utf8 : string) : string;
33. var temp:Pchar;
34. begin
35.    temp:=apl_command(pchar(command_utf8));
36.    setstring(Result,temp, length(temp));
37. end;
38.  
39. function apl_exec(line_utf8 : unicodestring) : integer;
40. begin
41.    Result :=apl_exec_ucs(PWideChar(line_utf8));
42. end;
43.  
44. function apl_command(command_utf8 : unicodestring) : unicodestring;
45. var temp:PWideChar;
46. begin
47.    temp:=apl_command_ucs(pWidechar(command_utf8));
48.    setstring(Result,temp,length(temp));
49. end;
50.  
51. end.

The UCS wrappers may be more comfortable since the APL syntax can be processed more natural.
I have overloaded them since that is more natural to Object Pascal.
(Note UCS is actually UCS2 which is a subset of Unicode16, limited to 16 bit codepoints)

[TRon]

I have edited post #17 to attach an updated zipfile for libapl.

I have re-arranged the examples, included some original documentation from APL trunk and added the two callback related functionality (with very simple invocation examples)

@Thaddy:
Thank you for the update. fwiw I've tried handlestream and got nowhere :-S

I am confused as strace clearly shows it using a handle with a value of 2. e.g. the redirection on stderr (as you did for stdout) should then in theory work ?

[Thaddy]

It *should* work. Will investigate. I will add the redirect options to the unit when written and tested.
Btw I link from the command line during development and use an external linker ld on most platforms except windows.

[marcov]

If you use commandline style redirection, either the library function you use needs to execute a shell (like system())) , or you have to specify a shell yourself.

Skimming posts can be risky, as is in this case 

Your input is appreciated but the used characters have nothing to do with redirection. It is how the APL language itself operates.

TS is trying to retrieve the output from the library (that is emitted to the terminal). I assume he wishes to do that so that the library can be used in a Lazarus GUI program.

I reacted to the Toby post immediately above it. Should have quoted.

Another tidbit that surfaced while reading through this thread: While I do work on TProcess from time to time, I am not that versed in *nix side of things anymore. But iirc some kinds of redirection there worked on glibc ( or c++ stdlib?)  level rather than on handle level. It is possible the C++ code does that.