capture apl_exec stdout

[toby]

hello

i can run the libapl apl_exec function in an fpc program but need to capture it's stdout and redirect to an output buffer instead of the screen and then covert into ansistring in the fpc program itself not as an external program which runcommand will do

this is example c++ code that does this

std::stringstream outbuffer;
std::streambuf *coutbuf = std::cout.rdbuf();
std::cout.rdbuf(outbuffer.rdbuf());
std::stringstream errbuffer;
std::streambuf *cerrbuf = std::cerr.rdbuf();
std::cerr.rdbuf(errbuffer.rdbuf());
execerr = apl_exec (cmd.toStdString ().c_str ());
std::cout.rdbuf(coutbuf);
std::cerr.rdbuf(cerrbuf);
outString = QString (outbuffer.str ().c_str ());
errString = QString (errbuffer.str ().c_str ());

'while apl_exec is running, standard output and standard error will be redirected to output and error buffers instead of the screen. These are then converted into strings.'

[Thaddy]

Simple example:

Code: Pascal  [Select][+][-]

1. program redirectstdout;
2. {$apptype console}
3. uses streamIO,classes;
4. var
5. f: TextFile;
6. s: TStringStream;
7. begin
8.   f:=stdout;
9.   s := TStringStream.Create;
10.   AssignStream(f, s);
11.   Rewrite(f);
12.   Writeln(f, 'Hello World');
13.   CloseFile(f);
14.   writeln('redirected: ', s.datastring);
15.   s.Free;
16. end.

But it is better to use Tprocess, simple example:

Code: Pascal  [Select][+][-]

1. {$mode objfpc}{$H+}
2. uses sysutils,classes, process;
3.  
4. var
5.  p:Tprocess;
6.  l:Tstrings;
7.  readcount:integer;
8.  a,b:string;
9.  i:integer;
10. begin
11.   p :=TProcess.Create(nil);
12.   try
13.     l := Tstringlist.create;
14.     try
15.       try
16.         P.Options := [];
17.         p.executable:='D:\fpctrunk\bin\x86_64-win64\ppcx64.exe';
18.         p.parameters.Add('-h');
19.         p.runcommandloop(a,b,i);// accumulate stdout in a, stderr in b
20.         l.add(a);
21.         l.add(b);
22.       except
23.         on E:Exception do writeln(E.Message);
24.       end;
25.       writeln(l.text);
26.     finally
27.       l.free;
28.     end
29.   finally
30.     p.free;
31.   end;
32.   readln;
33. end.

Side note: I was not aware of TProcess.RunCommandLoop until I wrote the example. It basically avoids messing with pipes yourself and greatly simplifies things. Because I was not aware of that, note you may need a fpc version 3.2.0 or higher, but then again it may have always been there.

If you have Pascal headers or C headers for libapl I can give a much better example, but when I try to find them I ran into many 404s

[toby]

Hi Thaddy,

your second example is for running an external program which is what i resorted to using my libapl/fpc coding but i need one fpc program that can run apl_exec and bring the results into the fpc program itself - it gets really complicaed having to maintain separate programs for each apl_exec line etc

i used your first code example and it worked for your example but the apl_exec stdout was not the screen output display but only the return value (0)

i would be glad to supply my fpc headers and c headers (libapl) but first do you have a working apl/libapl installed?

there are c/libapl and python3/lib_gnu_apl (and even lua  headers but i don't use them) included with the apl svn source

i compiled apl from svn source to get libapl/c and lib_gnu_apl/python3 headers but don't know if the distro versions have these done

i have c and python3 libapl programs using apl_exec that can test if you have a good apl/libapl installation

my fpc headers are simple and use the c/libapl headers and give good libapl/apl_exec for some very complicated apl expressions (the apl maintainer says that what i am doing shouldn't be doable with what he knows about the libapl - libapl is actually c headers for apl and separately authored from apl itself)

i can do everything i want with apl and fpc using external apl scripting libapl/fpc programs using runcommand but the libapl runs the same code almost 2x faster and

the libapl (whether fpc/c/python3) appears to be giving it's output to something other then stdout

i believe i am the only person working with fpc-3.2.2 and apl/libapl

---

[TRon]

the libapl (whether fpc/c/python3) appears to be giving it's output to something other then stdout

I/O channels depend on how things were build, see https://gist.github.com/houmei/cfd9e570b8de4d8fd55ada228d5ff004#file-readme-2-configure

i believe i am the only person working with fpc-3.2.2 and apl/libapl

Official GNU sources for apl do not compile on my setup (debian gcc 12.2). Official GNU apl .deb does not seem to include libapl.

[toby]

i have been asked if

> Does freepascal start foreign code in a separate process?  Without reopening stdout?

---

Hi TRon

I/O channels depend on how things were build, see https://gist.github.com/houmei/cfd9e570b8de4d8fd55ada228d5ff004#file-readme-2-configure

compiling libapl --with-android    made no difference-

-

'Official GNU sources for apl do not compile on my setup (debian gcc 12.2). Official GNU apl .deb does not seem to include libapl.'

this is what i thought was the situation

I would be glad to help you compile apl and libapl   i am on a 'debian' system with gcc-12.2.0 also

this is the svn line
svn co https://svn.savannah.gnu.org/svn/apl/trunk apl

you need to compile apl and libapl separately

for apl
configure --without-gtk3 --without-sqlite3 --without-postgresql --without-pcre
make
make install

type 'apl'  abd then  2+2    )off   to exit      you will need apl keyboard and apl font we can do them after things work

for libapl
configure --with-libapl --without-gtk3 --without-sqlite3 --without-postgresql --without-pcre
make
make install
check if /usr/local/lib/apl/libapl.[a la so] have been created
you need to edit /etc/ld.so.conf  and put the line   /usr/local/lib/apl     in it
ldconfig

this is example ;ibapl/c code

// libapl.c    compile with : g++ -O2 libapl.c -o libapl -L /usr/local/lib/apl -lapl
#include <stdio.h>
#include <stdlib.h>
#include <apl/libapl.h>
int main (int argc, char * argv[])
{
init_libapl(argv[0], 0);
apl_exec("2+2");
return 0;
}

i had to edit some source files to make libapl work - lets see if your system causes problems before we edit them

i have also edited files to change locations of default apl config files etc - lets see what happens with default compiles and installation and if you want to use what i have done

let me know how this goes

[TRon]

@toby: can't oblige right now. will try again as soon as time permits.

[toby]

this is the fpc header file i use - it give me complete access to all the apl commands with apl_exec - i left all the test lines (commented) that i have tried to no effect


unit libaplu;

interface

// extern void init_libapl(const char * progname, int log_startup);
procedure init_libapl(progname : ansistring; log_startup : integer); cdecl;

//extern LIBAPL_error apl_exec(const char * line_utf8);
function apl_exec(line_utf8 : ansistring) : longint; cdecl;
//function apl_exec(line_utf8 : pchar) : longint; cdecl;

//extern const char * apl_command(const char * command_utf8);
function apl_command(command_utf8 : pchar) : pchar; cdecl; // good : output is apl result
//function apl_command(command_utf8 : pchar) : longint; cdecl; // good : output is integer
//function apl_command(command_utf8 : pchar) : ansistring; cdecl; // does not worl
//function apl_command(command_utf8 : ansistring) : ansistring; cdecl; // does not work
//function apl_command(command_utf8 : ansistring) : longint; cdecl; // does not work

implementation

procedure init_libapl(progname : ansistring; log_startup : integer); cdecl; external;

function apl_exec(line_utf8 : ansistring) : longint; cdecl; external;
//function apl_exec(line_utf8 : pchar) : longint; cdecl; external;

function apl_command(command_utf8 : pchar) : pchar; cdecl; external;
//function apl_command(command_utf8 : pchar) : longint; cdecl; external;
//function apl_command(command_utf8 : pchar) : ansistring; cdecl; external;
//function apl_command(command_utf8 : ansistring) : ansistring; cdecl; external;
//function apl_command(command_utf8 : ansistring) : longint; cdecl; external;

end.

[Thaddy]

AnsiString functions need wrappers because it is a Pascal string type that apl does not know about, e.g:

Code: Pascal  [Select][+][-]

1. unit libaplu;
2. {$mode objfpc}{$H+ string=ansistring}
3. interface
4. { Pascal function wrappers }
5. procedure init_libapl(progname : string; log_startup : integer = 0);inline;
6. function apl_exec(line_utf8 : string) : integer;inline;
7. function apl_command(command_utf8 : string):string;inline;
8.  
9. { external declarations , no need to use them}
10. procedure init_libapl(progname : PChar; log_startup : longint);cdecl;external;
11. function apl_exec(line_utf8 : pchar) : longint; cdecl; external;
12. function apl_command(command_utf8 : pchar) : pchar; cdecl; external;
13.  
14. implementation
15.  
16. procedure init_libapl(progname : string; log_startup : integer);
17. begin
18.   init_libapl(Pchar(progname),log_startup);
19. end;
20.  
21. function apl_exec(line_utf8 : string) : integer;
22. begin
23.    Result :=apl_exec(PChar(line_utf8));
24. end;
25.  
26. function apl_command(command_utf8 : string) : string;
27. var temp:Pchar;
28. begin
29.    temp:=apl_command(pchar(command_utf8));
30.    setstring(Result,temp, strlen(temp));
31. end;
32.  
33. end.

I removed nonsense declarations.
Btw: you were lucky that your own init_libapl (AnsiString) worked!! I have corrected that.
Untested, but compiles and should be OK. The cdecl is not necessary for the wrappers.

Does anybody have a libapl.dll binary? (pref 64 bit, but 32 bit is acceptable)

[TRon]

Thank you for your header code Thaddy.

Does anybody have a libapl.dll binary? (pref 64 bit, but 32 bit is acceptable)

Does this satisfy your needs ?

@toby:
Thanks. I got trunk compiled now. libapl Is working as expected (even though I took a bit of another route but that is besides the point).

I am able to confirm that attempting to reroute as suggested by Thaddy's assignstream example (which was my first thought as well) does indeed not work as expected. If at all possible, I would need more time to figure out how.

[Thaddy]

Can you confirm my code works, the wrappers that is?

Thank you for your header code Thaddy.

Does anybody have a libapl.dll binary? (pref 64 bit, but 32 bit is acceptable)

Does this satisfy your needs ?

Alas no, did you get it to work? Seems only the keyboard driver.
Note I am missing some unitialize call. I am sure there must be one, because the dll or shared library uses its own memory management.
Can you post the signature for that? Then I can also add that one too.
If you experience any problems with my code plz inform here, but I am quite confident that it works.

About the console output: I probably need to change the APL sourcecode to be able to capture the handles.
But that is doable.

[TRon]

Can you confirm my code works, the wrappers that is?

Yeah wrappers work, (just some minor stuff see also rest of my answer)

Alas no, did you get it to work? Seems only the keyboard driver.

Sorry. I was not aware the zipfile only contained the keyboard driver. I simply noticed the dll's in there.

I did get libapl to compile and run as intended (on Linux) but alas the AssignStream was unable to catch the output from the library.

I assume this is because the c compiler's COUT/CERR/CIN and FPC's used handles are (internally) not using the same descriptors.

Sofar, I seem unable to solve that. As last resort it might perhaps be possible to redirect /proc/ IO filehandles but am not sure if that will work.

Note I am missing some unitialize call. I am sure there must be one, because the dll or shared library uses its own memory management.
Can you post the signature for that? Then I can also add that one too.

So far as I am able to see there simply is no uninitialization provided. library's include file header: https://svn.savannah.gnu.org/viewvc/apl/trunk/src/libapl.h?view=markup

If you experience any problems with my code plz inform here, but I am quite confident that it works.

The library itself, yeah that is no problem. I've made some minor correction to your (and toby's) code but nothing dramatic (e.g. parameter naming that reflects documentation, and I had to add the name of the library to the external modifier)

About the console output: I probably need to change the APL sourcecode to be able to capture the handles.
But that is doable.

It uses standard COUT/CERR/CIN.

[Thaddy]

Can you give us the modifications? By the end of the week we have a package  !

And yes, on linux everything works as expected (Debian and RaspberryOS tested, just Windows does not play nice).
I should be able to solve the IO issues. (and everything else in that header file, tnks)
It will be a nice addition to the standard packages.

I hope you did not touch the wrappers, though: that is pure pascalification and has almost no speed penalty because I inlined it.

[Fred vS]

If you want to capture all the output to terminal in Unix OS, you may use FpDup2

https://www.freepascal.org/docs-html/rtl/baseunix/fpdup2.html

Here example how to use it for redirect errors (you may redirect all stdout also, see fpc fpdup2 demo)

Code: Pascal  [Select][+][-]

1. program redirect-stdout;
2.  
3. uses
4.  BaseUnix;
5.  
6. var
7. fs: TFileStream;
8. ordir: string;
9.  
10. begin
11.  ordir := ExtractFilePath(ParamStr(0)) +
12.             directoryseparator + 'log.txt' ;
13.    
14.   fs := TFileStream.Create(ordir, fmOpenReadWrite or fmCreate);
15.  
16.   FpDup2(fs.Handle, StdErrorHandle);  
17.  
18.   // do your things ...
19.  
20.  application.run;
21.  
22.   fs.Free;
23.  
24. end.

[toby]

Thaddy

your libaplu.pas compiles and works fine   

i don't think there is an uninitialize call - (there is nothing in the /usr/local/include/apl/libapl.h and when working with the apl author on an error from not having init_libapl only needing the init_libapl was mentioned as missing from it

there is no unitialize in any of the c or python3 libapl code either

of interest : for full display when initializing libapl i use
init_libapl(paramstr(0), 1);

[toby]

Tron

in response to your post #10

Thaddy :
About the console output: I probably need to change the APL sourcecode to be able to capture the handles.
But that is doable.

Tron :
It uses standard COUT/CERR/CIN.

according to the link you gave in your post #3
https://gist.github.com/houmei/cfd9e570b8de4d8fd55ada228d5ff004#file-readme-2-configure

which says
-with-android
This option prevents the instantiation of CIN, COUT, and CERR. This maybe needed when GNU APL is compiled for Android, since Android applicationsare using a different I/O model than standard C++ programs.

when i compile my libapl --with-android and then my libapl program with new android libapli get
/usr/local/lib/apl/libapl.so: undefined reference to `CIN'
/usr/local/lib/apl/libapl.so: undefined reference to `UERR'
/usr/local/lib/apl/libapl.so: undefined reference to `COUT'
/usr/local/lib/apl/libapl.so: undefined reference to `CERR'

this is what i was told about these compiler errors
Those variables are C++ variables, so the names are probably mangled. You
can try to change the GNU APL code to declare them as extern "C".


i mistakingly said compiling --with-android made no difference (i forgot to copy the new libapl to /usr/local/lib/apl/libapl.so)  before compiling

would setting COUT to stdout be this solution for the stdout problem in the fpc code using libapl?