capture apl_exec stdout

[BobDog]

Windows console program.
catch stdout and stderr in an ansistring.

Code: Pascal  [Select][+][-]

1.  
2. {$APPTYPE CONSOLE}
3. uses
4. sysutils;
5.  
6. function  system(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
7. function tempnam(n:pchar;f:pchar):pchar ;cdecl external 'msvcrt.dll' name '_tempnam';
8.  
9. function filelength(filename:ansistring):longword;
10. Var F : File Of byte;
11. var L:longword;
12. begin
13. Assign (F,filename);
14.   Reset (F);
15.   L:=FileSize(F);
16.   Close (F);
17.   exit(L);
18. end;
19.  
20. procedure loadfile(var content: ansistring;filename:ansistring);
21.    Var Fin : File;
22.    Var x:longint;
23.    begin
24.    x:=filelength(filename);
25.    setlength(content,x);
26.    Assign (Fin,filename);
27.    Reset (Fin,x);
28.    BlockRead (Fin,content[1],1);
29.    close(fin);
30. end;
31.  
32.  
33. function stream(command:ansistring):ansistring;
34. var
35. _file:ansistring;
36. e:integer;
37. s:ansistring;
38. begin
39. _file:=tempnam(nil,'catchstream');
40. e:=system(pchar(command +' '+' >>'+_file +' 2>&1'));
41. loadfile(s,_file);
42. deletefile(_file);
43. if (FileExists(_file)=true) then writeln('delete '+_file+' manually');
44. exit(s);
45. end;
46.  
47.  
48. var
49. s:ansistring;
50.  
51.  
52. begin
53. s:=stream('fpc nosuchfile.pas');
54. writeln(s);
55.  
56. s:=stream('gcc -c nosuchfile.c');
57. writeln(s);
58. writeln('Press return to end . . .');
59. readln;
60. end.
61.  

[toby]

Tron

added to modified post
apl_exec_cb('rs1←⍳4', s); //  create rs1
//apl_exec_cb('⍕rs1', s); // good capture in s
//apl_exec_cb('⍞←rs1', s); // not captured in s ??
apl_exec_cb('⍞←⍕rs1', s); // hot captured in s ??
writeln('s : *', s, '*');

modified post info
the ⍕ character is the
Monadic format ⍕B A character representation of B U+2355 ⍕
which shows correctly on post createion using seaamonkey but as a 'bad unicode character' when i look at the post using dillo)

the callbacl method takes 96msec for a 30000 integer -> 89999 byte string test result (my largest current use exanple/need)
vs 26msec when writing the result to a file with apl and then reading it back into an tstringlist
(changing the call back apl_exec to apl_exec_cb did not effect anything time wises)

for reference the fred vs method of writing result to file and reading it back in also only takes 26msec

⎕pw ⍕ problem
this is not inlcuding the time that would be need to be taked to get rid of the #10 added to each ⎕pw defined line (i use ⎕pw←132) which i don't have to worry about with apl file writing method
⎕pw⊢80 gives 1250 lines vs ⎕pw←132 giving 715 lines
(i would have to use stringreplace( to remove the #10 chars
using ⍕ gives only 1 line - i am still playing with gettng the ⍕ to work with the call back but nothing so far.

(using ⍕ has no effect for some reason with the call back method as it doen with both file writing methods so there would be no ⎕pw line dependent #10 added

because of the 3x time increase i renamed your apl_exec call back function apl_exec(example, s) to apl_exec_cb(example. s)
and 'added' back in the original apl_exec for apl use and only use the apl_exec_cb callback for actualy capture of apl_exec output wnen needed.

also of major importance is that your apl_exec_cb code does not negatively impact using it with )copy command
i know i should use apl_command but i have automated things and testing every command in my command file for the one )copy command would be a waste of programming time

again thank you for you programming work here - you sure made it clean that the libapl coding needs rewriting but i fear the original libapl dev is long gone

this has been fun coding for me indeed

[TRon]

catch stdout and stderr in an ansistring.

Close but no cigar 

Thank you for your effort BobDog but the issue is about catching your own program's I/O. Or to be more precise: the I/O from a library that is used in your program. And it already works (for stdout) except that it seems impossible to do the same for stderr.

[TRon]

//apl_exec_cb('⍞←rs1', s); // not captured in s ??
apl_exec_cb('⍞←⍕rs1', s); // hot captured in s ??

Those are using an immediate input request. For that you need to use/install a callback with install_get_line_from_user_cb.

I've added functionality for that to reply #17 on jan 17th, see also example apl_test2b.pas

You are on your own with regards to the different input modes as I have no clue how the different modes should behave (I simply lack the time to investigate as documentation for apl for/on certain topics is very scarce).

the ⍕ character is the Monadic format ⍕B A character representation of B U+2355 ⍕

the callbacl method takes 96msec for a 30000 integer -> 89999 byte string test result (my largest current use exanple/need) vs 26msec when writing the result to a file with apl and then reading it back into an tstringlist

Can't comment on that a.t.m. other then If you need a different storing method for the result (f.e. stringlist) then you can create your own callback wrapper function.

I have no idea w.r.t. the time(penalty). You should be looking at direct library function calls, see also below.

⎕pw ⍕ problem

Are you referring to the plot window functionality ? (e.g. apl is non-case sensitive so that pw = PW ?)

this is not inlcuding the time that would be need to be taked to get rid of the #10 added

Although there is little documentation w.r.t. behaviour on certain functionality /this/ is at least a topic that is present in the docs 

Have a look at CR. It is advised to use that in combination with certain function(s)/(ality)

There is an example in the official documentation that removes them i.c.w. XML.

because of the 3x time increase i renamed your apl_exec call back function apl_exec(example, s) to apl_exec_cb(example. s) and 'added' back in the original apl_exec for apl use and only use the apl_exec_cb callback for actualy capture of apl_exec output wnen needed.

In that regards you can do whatever you please It is encouraged to write your own additions/correction. Only then things can improve

Just keep in mind that when you run into problems that you post the complete implementation that you used because how you implement things can influence behaviour (which we cannot replicate without using the same implementation).

I've named it the way I did for convenience because it is an overloaded function, which keeps things easy for the end-user.

Also note that I set and remove the callback for each call to apl_exec because you can install a different callback based on what functionality you require (and that can depend on which function you are actually executing).

So if you need other functionality then go ahead and add it to your own custom callback routine and name it whatever floats you boat 

also of major importance is that your apl_exec_cb code does not negatively impact using it with )copy command

You should have a better/closer look at the other functionality that the apl library exposes.

I simply lack the time to delve deeper into things but am able to tell you that you can call functions directly as well as access values using direct calls to functions.

I'm a bit stuck with regards to the loc parameter stuff. Seems highly undocumented so would need to delve into their source-code to see how that should be used correctly.

Also I am unsure about how custom functions affect the different workbook environments when using direct library calls.

If you are interested in apl then you should experiment with those as well. I have added them in my local copy of ulibapl.

My knowledge about the apl language/library is pretty limited so am more often not able to tell if the experienced behaviour is as expected.

regards,

[toby]

Hi Tron

there really is no problem really with anything you did - i was just getting into the weeds

i have gotten the call back time decreased by doing more inline apl coding

so this is great - no problem with any apl functions or apl commands and all my code runs as expected

see Dr post where use of ⍕ for apl line submitted to the call back is needed - i have just included a ⍕ before the apl_line before submitted to the call back

---

i took care of my ⎕pw problem

the ⎕ and ⍞  apl input/output methods with ⍕ are use dependent
with ⎕pw they are like
writeln vs write with an extra writeln thrown in after the ⎕pw number of chars

var i, j : integer;

begin

for i := 1 to 9 do // ⎕pw := 9
  begin
  for j := 1 to 9 do write(i, j, ' ');
  writeln; //
  end;

end.

if you do do any additional programming to your ulibapl.pas could you please add it as a new reply to this forum/topic thread instead of as a modified post?  thanks

it is easier to check for new replies rather than if a post has been modified

for very very large captures the time taken tapers off - there must be a minimum time overhead to it ?

you never mentioned it but did you have any problems using the apl keyboard/fonts ?
i added the following to an    apl.xodmap file so with xmodmap all the apl key combo are on the right alt key and the everything programmed for fluxbox and dillo and .... etc are left alone on the left alt key
keycode 113 = Mode_switch ISO_Level3_Shift Multi_key

[BobDog]

catch stdout and stderr in an ansistring.

Close but no cigar 

Thank you for your effort BobDog but the issue is about catching your own program's I/O. Or to be more precise: the I/O from a library that is used in your program. And it already works (for stdout) except that it seems impossible to do the same for stderr.

I can catch output from a pascal file along with errors, hints, or any other information needed, both from stdout and stderr using my previous code.
I really could use a cigar but I have quit smoking, doctor's orders.
Example

Code: Pascal  [Select][+][-]

1.  
2.  
3.  
4. {$APPTYPE CONSOLE}
5. uses
6. sysutils;
7.  
8. function  system(s:pchar):integer ; cdecl external 'msvcrt.dll' name 'system';
9. function tempnam(n:pchar;f:pchar):pchar ;cdecl external 'msvcrt.dll' name '_tempnam';
10.  
11. function filelength(filename:ansistring):longword;
12. Var F : File Of byte;
13. var L:longword;
14. begin
15. Assign (F,filename);
16.   Reset (F);
17.   L:=FileSize(F);
18.   Close (F);
19.   exit(L);
20. end;
21.  
22. procedure loadfile(var content: ansistring;filename:ansistring);
23.    Var Fin : File;
24.    Var x:longint;
25.    begin
26.    x:=filelength(filename);
27.    setlength(content,x);
28.    Assign (Fin,filename);
29.    Reset (Fin,x);
30.    BlockRead (Fin,content[1],1);
31.    close(fin);
32. end;
33.  
34. procedure savefile(s:ansistring ;filename:ansistring);
35.     var
36.     fout:file;
37.     begin
38.     Assign(fout,filename);
39.     Rewrite(fout,length(s));
40.     blockwrite(fout,s[1],1);
41.     close(fout);
42.   end;
43.  
44.  
45. function stream(command:ansistring):ansistring;
46. var
47. _file:ansistring;
48. e:integer;
49. s:ansistring='';
50. begin
51. _file:=tempnam(nil,'catchstream');
52. e:=system(pchar(command +' '+' >>'+_file +' 2>&1'));
53. loadfile(s,_file);
54. deletefile(_file);
55. if (FileExists(_file)=true) then writeln('delete '+_file+' manually');
56. exit(s);
57. end;
58.  
59.  
60. var
61. s:ansistring;
62. txt:ansistring;
63.  
64.  
65.  
66.  
67. begin
68. txt:='{$APPTYPE CONSOLE}'+#10;
69. txt:=txt+'var'+#10;
70. txt:=txt+'i,j:integer;'+#10;
71. txt:=txt+'begin'+#10;
72. txt:=txt+'for i:=1 to 20 do writeln(i);'+#10;
73. txt:=txt+'end.'+#10;
74.  
75. savefile(txt+chr(10),'vtemp.pas');
76.  
77. s:=stream('type vtemp.pas');
78. s:=s+stream('fpc -vwnhi vtemp.pas');
79. s:=s+stream('vtemp.exe');
80.  
81. savefile(s,'results.txt');
82.  
83. system('notepad.exe results.txt');
84. writeln('Press return to end . . .');
85. readln;
86. deletefile('vtemp.pas');
87. deletefile('vtemp.exe');
88. end.
89.  

[TRon]

I can catch output from a pascal file along with errors, hints, or any other information needed, both from stdout and stderr using my previous code.

The code that you showed (thank you for the example) is executing an external program and catches that program's error/output, not its own error and output. You also combined out and err which is a no-go for using this 3th party library. TS also expressed not wanting to use external files.

All the above was already mentioned in this thread.

fwiw: Your code looks overly complicated to me for something that can be done in a cross-platform manner with runcommand

I really could use a cigar but I have quit smoking, doctor's orders.

Smart doctor, smart patient 

[TRon]

Just a quick note, i'll try respond to your other remarks later.

if you do do any additional programming to your ulibapl.pas could you please add it as a new reply to this forum/topic thread instead of as a modified post?  thanks

I have no problem posting my additions and will do so later. It is just that my code includes dynamic loading which isn't supported by the library so I have to remove that and i haven't had the time to do that yet.

it is easier to check for new replies rather than if a post has been modified

Actually I disagree on that matter. That way I would have to keep track of every post with a published zip in the thread to make sure old attachments are removed. As things progresses, so  will the length of the thread so that people have to read through the whole thread to be able to locate the latest attachment. imo keeping old attachments alive works counterproductive as people tend to download old versions then ask about issues that are already solved in a later version and keeping old attachments will needlessly waste forum space.

Thus I believe it is more productive/maintainable to post whenever a new update is available and point to the post that includes the (replaced and updated) attachment. Ideally that would be the first post of the thread but alas that ship has sailed 

[TRon]

Update post #17 and attached version v3.

I have added a license which I deemed to be satisfactory (pay it forward) but since the package also contains work from others and/or is based on suggestion/examples made by others from this forum i have to take into consideration that someone would object to that.

In such case please let me know so I can rectify anything you do not agree with/to.

Mentioned users in package:
- Thaddy (for his contributions/ideas)
- FredVs (for his ideas/examples)
- Engkin (for his piping example code somewhere on this forum which inspired me how a solution for libapl could be implemented)

[toby]

Tron

looks like you have no problem taking my original libapl work and appropriating it as your own

pretty sad

[TRon]

Tron

looks like you have no problem taking my original libapl work and appropriating it as your own

pretty sad

[TRon]

APL Library itself published under the GNU General Public License

[TRon]

All Pascal related files in this archive are licensed as "Pay it forward"

[TRon]

Tron

looks like you have no problem taking my original libapl work and appropriating it as your own

pretty sad

@toby:

Please accept my deepest apologies as I humbly ask you to forgive me for my rude manners in copy-pasting all your hard work and claim it as my own (wherever I supposedly did that, thus please enlighten me).

Would you be so kind and point me to the work that you designated as your "original libapl work" so that I am able to rectify my blameworthy behaviour ?

As a (demonstrable) reminder: All my work, with the exception of users mentioned in post #68 (as well as in the source-files) is based on publications from libapl and/or is actually my own.

[toby]

you want to show me where a libaplu.pas was in any apl publications or online??
there was NO libaplu.pas until i wrote it and submitted it as reply #6 to this topic

NOTHING you did has increased the usable functionality of my libaplu.pas - NOTHING
nothing you did in your ulibapl.pas is needed to get the most complex apl/apl_exec statements running - NOTHING

in fact i have been able to bring the results of an apl_exec statement into an fpc ansistring and tstinglist for years

your call back increases the time to get the rsults into a tstiringlist and ansistring to unacceptable amounts

---

timing testing :  run on your system to verify - use your apl_exec callback     compare for relative times from my system to yours
also note the bug in the results your callback and fpdup methods produce

Date Friday 2/17/2023 Time 13:58.46.298         20 msec to create rs with 1 line of 900000 chars
rs←900000↑"ab "
⍴rs
900000
Date Friday 2/17/2023 Time 13:58.46.318

Date Friday 2/17/2023 Time 13:58.46.318
soa rs                      45 msec for my apl fns to bring rs into an fpc tstringlist and then ansistring s
astringlist.count : 1       correct nymber of lines of 900000 chars
s := astringlist.text;
length(s) : 900000          correct number of chars
Date Friday 2/17/2023 Time 13:58.46.363


Date Friday 2/17/2023 Time 13:58.46.363
so rs                      10 seconds 239 msec for fpdup2 method to bring rs into an foc tstringlist and then ansistring s
astringlist.count : 12163        it should be 1 line of chars not 12163 lines
length(s) : 985134               it should be 900000 chars
Date Friday 2/17/2023 Time 13:58.56.602


Date Friday 2/17/2023 Time 13:58.56.602
soc rs          11 seconds 862 msec fpr your call back code to bring rs into an fpc tsringslit and then ansistring s
astringlist.count : 12163        it should be 1 line of chars not 12163
length(s) : 985135               it should be 900000 chars
Date Friday 2/17/2023 Time 13:59.8.482

run bringing the rs apl variable into an fpc taringlist and assistring from the fpdup and call back methods on your system and post the timing results


your problem is even more pronouned with a simple
rs←⍕3 3000⍴⍳9
which should resukt in rs being a 3x3000 array     3 lines of 3000 '9'
your code and the fpdup code produce result of 243 lines and different lengths if s
in order to fix your code to give correct reaults the time was increased from and already unacceptable time