Recent

Author Topic: Console In/Output slow  (Read 4680 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9663
  • Debugger - SynEdit - and more
    • wiki
Re: Console In/Output slow
« Reply #45 on: February 12, 2024, 03:38:27 pm »
I understood you, that your patch is included in *Lazarus* trunk, so I installed Lazarus trunk = main_3_99-1434-g76da2aad75 with FPC 3.2.2-r0d122c49 (release).

With FPDebug I see only 461 lines. If I switch to GDB, I see 1000 lines. So this patch does not work here.

As said, I'm on Linux Ubuntu 22.04 64-bit with GTK2.

Yes you only need Lazarus trunk/main.

Strange, since I could reproduce without the patch, but no longer with it (Fedora).

Do you have multiple Lazarus installed? Just to be sure there was no mixup? Also could you check if the file content matches the link I gave to gitlab?
Sorry for the extra inconvenience.

main_3_99-1434-g76da2aad75 is definitely correct

zeljko

  • Hero Member
  • *****
  • Posts: 1582
    • http://wiki.lazarus.freepascal.org/User:Zeljan
Re: Console In/Output slow
« Reply #46 on: February 12, 2024, 03:45:30 pm »
Tested Lazarus 3.99 (rev main_3_99-1412-gd6b7ffd24c) FPC 3.2.2 x86_64-linux-qt5 LinuxMint 20.02 - everything works fine, even 50k writelns are written to the console in/out (with fpdebug and with gdb).

Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Console In/Output slow
« Reply #47 on: February 12, 2024, 04:31:43 pm »
Do you have multiple Lazarus installed? Just to be sure there was no mixup?

I have multiple Lazarus installed. The new installation was made via fpcupdeluxe version 2.4.0c into a new, empty folder. My test was with all IDE-options as Defaults.

Quote
Also could you check if the file content matches the link I gave to gitlab?

I checked your patch = https://gitlab.com/freepascal.org/lazarus/lazarus/-/commit/47c34d2e7ea0356f0b101becc5df8cc311e8c4cd (from reply #31) manually and what I see, the patch is there. This is my file <installdir>/lazarus/components/fpdebug/fpdbglinuxclasses.pas from line 1434 on:
Code: Pascal  [Select][+][-]
  1. function TDbgLinuxProcess.GetConsoleOutput: string;
  2. var
  3.   ABytesAvailable: DWord;
  4.   ABytesRead: cint;
  5.   s: string;
  6. begin
  7.   if fpioctl(FMasterPtyFd, FIONREAD, @ABytesAvailable)<0 then
  8.     ABytesAvailable := 0;
  9.  
  10.   result := '';
  11.   while ABytesAvailable>0 do
  12.   begin
  13.     setlength(s, ABytesAvailable);
  14.     ABytesRead := fpRead(FMasterPtyFd, s[1], ABytesAvailable);
  15.     SetLength(s, ABytesRead);
  16.     Result := Result + s;
  17.  
  18.     if fpioctl(FMasterPtyFd, FIONREAD, @ABytesAvailable)<0 then
  19.       ABytesAvailable := 0;
  20.   end;
  21. end;

As an additional test I set up a new project within Lazarus 3.99 to have virgin project options. A question regarding the debugger info format (see screenshot) I answered with the default (Dwarf 3).
I attached this project, maybe this helps you.

jcmontherock

  • Full Member
  • ***
  • Posts: 223
Re: Console In/Output slow
« Reply #48 on: February 12, 2024, 05:09:52 pm »
Did you try the "Keyboard" package ?
Windows 11 UTF8 64 - Lazarus 3.0 64 - FPC 3.2.2

Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Console In/Output slow
« Reply #49 on: February 12, 2024, 05:19:30 pm »
Did you try the "Keyboard" package ?
No, I did not install any packages. Should I? I tested with a virgin installation made by fpcupdeluxe.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9663
  • Debugger - SynEdit - and more
    • wiki
Re: Console In/Output slow
« Reply #50 on: February 12, 2024, 05:22:38 pm »
Did you try the "Keyboard" package ?
How is that related?????????????????????

jcmontherock

  • Full Member
  • ***
  • Posts: 223
Re: Console In/Output slow
« Reply #51 on: February 12, 2024, 09:02:47 pm »
... To the first question of the topic !
Windows 11 UTF8 64 - Lazarus 3.0 64 - FPC 3.2.2

jollytall

  • Sr. Member
  • ****
  • Posts: 301
Re: Console In/Output slow
« Reply #52 on: February 13, 2024, 09:18:58 am »
Which first question do you mean?
As I mentioned, I did not add any special package, anyway.
Without adding any package, of course I can simply add a readln; also instead of a sleep(), so the problem is not that I cannot get the whole printout, the problem is that it should not work like that (as the long thread above details all aspects of it).

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9663
  • Debugger - SynEdit - and more
    • wiki
Re: Console In/Output slow
« Reply #53 on: February 13, 2024, 10:28:48 pm »
Did you try the "Keyboard" package ?
No, I did not install any packages. Should I? I tested with a virgin installation made by fpcupdeluxe.

With regards to the issues of the "console output" not showing all info => no. no need to install any package.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9663
  • Debugger - SynEdit - and more
    • wiki
Re: Console In/Output slow
« Reply #54 on: February 13, 2024, 10:48:10 pm »
Ok, one of my other VM also kept being incomplete after the patch.

It seems it is a timing/waiting issue.

With GDB, GDB sends the message that the target app stopped to the IDE via a pipe. However fast pipes may be, it takes a moment (and the same moment may be needed by the StdOut of the target app to pass through its pipe).
So the content is read, even if a short delay is needed.

FpDebug is too fast. (really that shouldn't be a bad thing...)

So I added a "sleep"
https://gitlab.com/freepascal.org/lazarus/lazarus/-/commit/972e988e00b4ee0c361d882ec5f19abf9e246e33

For me, even a sleep(20) was enough. But that can depend on a ton of things. I added sleep(50) in the hope it is enough.
If it is not, please try to increase it in steps of 100, until you get all the content.


I have merged this to fixes branch now. Since it definitely improves the situation on some platforms.

I may look at moving the code, so that the debugger can (in OnIdle) keep reading for another few seconds. Or maybe it can detect if the pipe got closed.

But for now, I wanted something that did not change a lot of code, so it could easily be merged to the fixes branch. (If needed the sleep can still be increased a bit).

Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Console In/Output slow
« Reply #55 on: February 14, 2024, 01:27:08 pm »
I installed a new trunk version and tested but unfortunately it does not work:

Lazarus = main_3_99-1455-g760247b8d1
FPC = 3.2.2-r0d122c49 (release)
OS = Linux Ubuntu 22.04 64-bit with GTK2

On the 1st test I kept the loop 'for i:=1 to 1000 do' and I got only 233 and a half lines (see screenshot1) and the IDE hung (was not longer operable and must be restarted).

Several tests with loop values up to 230 worked perfectly (see screenshot2).

All loop values from 235 and greater failed:

Several tests with loop values from 235 to 1000 showed NO Output (empty output window) and the IDE always hang.

During 4 tests with loop value=2000 I 3x saw 1136 lines (see screenshot3) and the IDE hang. 1x I saw 1351 lines and the IDE hung.

All tests have been with all IDE-options as Defaults after the new installation (see screenshot4).

I checked your patch = https://gitlab.com/freepascal.org/lazarus/lazarus/-/commit/972e988e00b4ee0c361d882ec5f19abf9e246e33 (from reply #54) manually and what I see, the patch is there. This is my file <installdir>/lazarus/components/lazdebuggers/lazdebuggerfp/fpdebugdebugger.pas from line 3474 on:

Code: Pascal  [Select][+][-]
  1. procedure TFpDebugDebugger.FDbgControllerProcessExitEvent(AExitCode: DWord);
  2. var
  3.   AThread: TFpWaitForConsoleOutputThread;
  4. begin
  5.   if assigned(FConsoleOutputThread) then
  6.     begin
  7.     AThread := TFpWaitForConsoleOutputThread(FConsoleOutputThread);
  8.     FConsoleOutputThread := nil;
  9.     AThread.Terminate;
  10.     AThread.WaitFor;
  11.     sleep(50);
  12.     AThread.DoHasConsoleOutput(0);
  13.     AThread.Free;
  14.     end;
  15.  
  16.   SetExitCode(Integer(AExitCode));
  17.   {$PUSH}{$R-}
  18.   DoDbgEvent(ecProcess, etProcessExit, Format('Process exited with exit-code %u',[AExitCode]));
  19.   {$POP}
  20.   LockRelease;
  21.   try
  22.     SetState(dsStop);
  23.     StopAllWorkers;
  24.     FreeDebugThread;
  25.   finally
  26.     UnlockRelease;
  27.   end;
  28. end;

Thaddy

  • Hero Member
  • *****
  • Posts: 13938
  • Probably until I exterminate Putin.
Re: Console In/Output slow
« Reply #56 on: February 14, 2024, 01:38:53 pm »
i am missing a Output.Flush() in your code, or on the outputstream itself
« Last Edit: February 14, 2024, 01:41:27 pm by Thaddy »
Specialize a type, not a var.

Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Console In/Output slow
« Reply #57 on: February 14, 2024, 01:52:25 pm »
i am missing a Output.Flush() in your code, or on the outputstream itself

You can see 'flush(Output)' in screenshots 2 and 3 in above post.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9663
  • Debugger - SynEdit - and more
    • wiki
Re: Console In/Output slow
« Reply #58 on: February 14, 2024, 02:24:26 pm »
All loop values from 235 and greater failed:

Several tests with loop values from 235 to 1000 showed NO Output (empty output window) and the IDE always hang.

During 4 tests with loop value=2000 I 3x saw 1136 lines (see screenshot3) and the IDE hang. 1x I saw 1351 lines and the IDE hung.

For the test with cut off output (some output, but not all): Can you try with "sleep(150)" or "sleep(250)" .... "sleep(1550)" ?




The IDE hanging is odd....

If that is related to the patch, it would have to be
Code: Pascal  [Select][+][-]
  1. function TDbgLinuxProcess.GetConsoleOutput: string;
  2. var
  3.   ABytesAvailable: DWord;
  4.   ABytesRead: cint;
  5.   s: string;
  6. begin
  7.   if fpioctl(FMasterPtyFd, FIONREAD, @ABytesAvailable)<0 then
  8.     ABytesAvailable := 0;
  9.  
  10.   result := '';
  11.   while ABytesAvailable>0 do
  12.   begin
  13.     setlength(s, ABytesAvailable);
  14.     ABytesRead := fpRead(FMasterPtyFd, s[1], ABytesAvailable);
  15.     SetLength(s, ABytesRead);
  16.     Result := Result + s;
  17.  
  18.     if fpioctl(FMasterPtyFd, FIONREAD, @ABytesAvailable)<0 then
  19.       ABytesAvailable := 0;
  20.   end;
  21. end;
  22.  

fpioctl either hangs.... In which case it probably is a race condition that existed before the change too.

Or "ABytesAvailable" never becoming 0. 


Maybe your distro has an ioctl implementation that differs... (strange though, since it works for other amounts of data).

Could you please do a test, please.

Add a counter to the above loop, and abort it at 100 iterations.
Code: Pascal  [Select][+][-]
  1. TESTCNT := 0; ///// <<<<
  2.   while (ABytesAvailable>0)    AND (TESTCNT < 100)    do
  3.   begin
  4.     inc(TESTCNT); // <<<<<<<<<<<<<<<<
  5.     setlength(s, ABytesAvailable);
  6.  

Then also check, if the output has any unexpected data....

If that still hangs, the fpIoctl probably hangs. Assuming it is not something entirely different.

If you can run the IDE itself in the debugger please.


Hartmut

  • Hero Member
  • *****
  • Posts: 736
Re: Console In/Output slow
« Reply #59 on: February 14, 2024, 02:56:01 pm »
I verified that your patch in <installdir>/lazarus/components/fpdebug/fpdbglinuxclasses.pas is in my newly installation exactly as you posted in reply #58.

I will do the requested tests for you. Do I see this right, that I have to recompile the IDE after changing the sleep value and after inserting counter 'TESTCNT' (I'm not familiar with that)?

If yes, to recompile the IDE I use menu "Tools / Build Lazarus with Profile: Normal IDE" and that's all?

 

TinyPortal © 2005-2018