Console In/Output slow

[Thaddy]

The flush belongs to the thread.... I already told you so.

[jollytall]

Well, I missed that. In your first replies there was no word of threads at all. Later it was about the end of the thread, "especially with multiple threads". However to me the main thread is also a thread, even if it is not an "especially" case. So, it was not clear to me that you meant only multi-threading, particularly, because that was not even mentioned in the original problem.
Anyway, I tried and it did not do any harm, neither good though.

[Martin_fr]

Runcommandloop polls the pipes again after the process is terminated to make sure.

That is TProcess?  The debugger (both fpdebug and gdb) don't use the processes internal pipes. In case of gdb that would be the communication with gdb anyway. In both cases, the output of the child process is redirected to a virtual pty.

And that is then polled inside the debugger-backend.

But yes there is one last poll in FpDebug, after the app exited.

Code: Pascal  [Select][+][-]

1. procedure TFpDebugDebugger.FDbgControllerProcessExitEvent(AExitCode: DWord);
2. ...
3.     AThread.DoHasConsoleOutput(0);
4.  

Code: Pascal  [Select][+][-]

1. procedure TFpWaitForConsoleOutputThread.DoHasConsoleOutput(Data: PtrInt);
2. ...
3.     s := FFpDebugDebugger.FDbgController.CurrentProcess.GetConsoleOutput;
4.  

Code: Pascal  [Select][+][-]

1. function TDbgLinuxProcess.GetConsoleOutput: string;
2. ...
3.   if fpioctl(FMasterPtyFd, FIONREAD, @ABytesAvailable)<0 then
4. ...
5.     ABytesRead := fpRead(FMasterPtyFd, result[1], ABytesAvailable);
6.  

That does leave 2 questions though.

1) How fast does the pty work?
2) Does such a read guarantee to read everything available, or should it be called several times until no further data was read.

One other test would be to add a sleep into the code above FDbgControllerProcessExitEvent before the call to do the read.
That would test if there is any delay.

If only part of the data may be read, then a loop in DoHasConsoleOutput ( "repeat ... until (data <> 0) or (s = ''); " )


On the "thread" discussion....

My reading about the original post by jollytall is that all references to "thread" are assumptions of threads running in the Lazarus IDE?
And that the app that jollytall runs does not itself uses threads (or if it does only his main thread does interact with StdOut)?

@jollytall Can you explicitly confirm this? (I am not sure about your last reply on threading).

[MarkMLl]

That does leave 2 questions though.

1) How fast does the pty work?
2) Does such a read guarantee to read everything available, or should it be called several times until no further data was read.

Assuming that we are discussing Linux here: in general, a pty is fast and is used heavily by e.g. ssh. It can, however, be slowed down substantially by the GUI, which is why there's a line limit.

As I've said earlier, fragmentation etc. is a problem and things like \r\n more often than not get split. I presume that's something you're aware of, and it was some aspect of that and/or the transition from gdb to fpdebug that stopped some of the "decorations" I added working.

MarkMLl

[jollytall]

I can confirm that no threading is involved in my program at all. See my two line for loop above. That is it, nothing else and the issue is there.
Also, there is no debugging involved in this case.

I made some more tests:
- Removing any debugging (Project Options/Compiler Options/Debugging/Generate info for debugger unchecked) does not change the number of lines printed.
- Changing the optimization level does not change anything (well, for this "complex" program I guess there is not much optimization anyway).
- Today, a new day, after the PC was switched off, different programs run now, etc., but still 461 lines are printed. It seems that it is deterministic, not a simple timing issue.
- However if I change even one character in the printout, it changes. Instead of 'My number is '  I use 'My number is' then I get 488 lines. Nonetheless it is not that longer text means less lines. If I use 'My number is = ' then I get 619 and half a line.

[MarkMLl]

Try changing your code so that it looks like

Code: Pascal  [Select][+][-]

1.     var i : integer;
2.     begin
3.     {$I-}
4.     for i:=1 to 1000 do
5.       writeln(Output, 'My number is ', i);
6.     flush(Output);
7.     end.
8.  

i.e. you're explicitly referring to the same output device rather than relying on the default.

Other than that... I'm not aware of any output problems other than those I've discussed on Lazarus v2 built for gtk2 with Debian "Bookworm" x86_64 and KDE.

MarkMLl

[jollytall]

Nope. It is the same problem, with exactly the same number of lines.

Has anyone else tried it and see the same, or is it only me?

[MarkMLl]

In OP you say "I am using Linux (Debian, gtk3) and Lazarus 2.2.6."

Do you mean that Lazarus is built for GTK3 or that Debian is using something based on GTK3 for its desktop environment?

In the IDE, View -> IDE Internals -> About IDE and please paste the first para of output which should be the same as the IDE's About box.

Where did you get the IDE, and are there any unusual plugins etc.?

Can we have your /entire/ program in a compilable state?

MarkMLl

[Hartmut]

Has anyone else tried it and see the same, or is it only me?

Because I don't have Lazarus 2.2.6 I tried your demo from reply #14 with Lazarus 2.2.4 / FPC 3.2.2 on Linux Ubuntu 22.04 64-bit with GTK2 and there is no problem, I get all 1000 lines (see screenshot).

[Martin_fr]

Because I don't have Lazarus 2.2.6 I tried your demo from reply #14 with Lazarus 2.2.4 / FPC 3.2.2 on Linux Ubuntu 22.04 64-bit with GTK2 and there is no problem, I get all 1000 lines (see screenshot).

Using gdb or fpdebug?

[Hartmut]

Using gdb or fpdebug?

I'm not familiar with this question. As far as I can see I'm using gdb. I attached 2 screenshots. Does this answer your question?

[marcov]

But yes there is one last poll in FpDebug, after the app exited.

Code: Pascal  [Select][+][-]

1. procedure TFpDebugDebugger.FDbgControllerProcessExitEvent(AExitCode: DWord);
2.  

That was what it was my remark wa sabout. I had some problems with that, which is why I said it.

I'm also not that deep into Linux process execution (beyond the basics), so I'll be quiet now :-)

[Martin_fr]

Yes, you do use gdb. And if jollytall might use FpDebug then the experience may be affected by this.

Btw, off topip, but

1)
"Additional search path" https://wiki.lazarus.freepascal.org/IDE_Window:_DebuggerGeneralOptionsFrame (should open on F1)

If the IDE needs to find the sources

"sources" => pascal sources that are not in the project folder, or any folder configured by the project, nor any folder of a package used (in other words sources you added by manually specifying them to the compiler, bypassing the IDE).

So "/usr/bin/gdb" wont help here.

2)
If you have an Intel/Amd cpu, then on the backend (your 2nd image) you may consider trying "FpDebug internal Dwarf debugger".  For most people it gives a much better debugging experience. (even in 2.2.6 / certainly in 3.0)
Except of course, potentially for the problem reported here on the "cut off" output window (if it turns out to be caused by FpDebug)

[jollytall]

@MarkMLI

The Lazarus IDE Internal first lines are:
Lazarus version: 2.2.6
Lazarus revision: Unversioned directory
Lazarus build date: 2023/06/20
Lazarus was compiled for x86_64-linux
Lazarus was compiled with FPC 3.2.2

As far as I remember I only added from the on-line package manager laz_fpspreadsheet, but I don't use it in this project. I am not aware of any other specialty. I downloaded the IDE from a standard place (do not remember where), but nothing special.

The gtk3 I found in another software and thought it might be across the platform, but probably I was wrong, so if it is irrelevant, please omit.

The compilable program, is as shown above:

Code: Pascal  [Select][+][-]

1. program loop;
2.  
3. var i : integer;
4. begin
5. for i :=1 to 1000 do
6.   writeln('My number is ', i);
7. end.

and the result is in the first attachment. If I change the writeln to ('My number is = ', i) then the result is in the second attachments (please notice, that it is more lines, although the writeln is longer and the last line is a half line)

[Martin_fr]

Just tested, it only happens with FpDebug. Gdb based backend works.

So it is a bug in the IDE => please report.