Recent

Author Topic: Lazarus/FPC semi-randomly gets stuck while compiling  (Read 3029 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #15 on: November 12, 2024, 11:19:28 am »

You can also watch the messages windows: Does it say anything about "compiling package ..."? Or does it only output stuff about compiling your project?

Because, if LCL and dependencies is not recompiled, and if you have no other packages.... And if I am right, then there should be only one instance of fpc and one of ppc. (Someone of the fpc team might be able to confirm that).


I just got word from a member of the fpc team. If really only your project is buid, then there should be only one fpc and one ppc process (and both are single threaded).

That is, if the IDE only starts one compilation. Which in this case it should.

You don't have any custom settings in "complier commands" of the project settings?


Thaddy

  • Hero Member
  • *****
  • Posts: 16174
  • Censorship about opinions does not belong here.
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #16 on: November 12, 2024, 12:20:10 pm »
The problem with the thread locks is:
- indeed you can compile lazarus as a whole divided over several threads, I believe it is even the default.. And multiple ppc's as a result.
- but packages can rely on shared units and resources, like sysutils, objpas, etc, etc, etc, even ppu's.
- and such constructs can subsequently cause a thread deadlock issue.
It is not something that happens often but it does happen every now and then, that's why I turned that off.
Building Lazarus with several threads does not always work very well. If it works, it works much faster, though. 9 out of 10 cases or better.
It is a bad luck = bad lock issue. Building FPC itself does not suffer from that issue.
It is darn hard to debug, so I chose the easy way out.
You can check that this is indeed the case in process windows on several platforms like windows and linux.
« Last Edit: November 12, 2024, 12:47:12 pm by Thaddy »
If I smell bad code it usually is bad code and that includes my own code.

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #17 on: November 13, 2024, 03:48:18 pm »
If you also did not add a path in your "Project options" settings on the "Path" page, then that is fine.
If you did put any path there that points into the Lazarus or FPC sources, or unit or lib folders then that may be an issue.
If you did put a path there that points some other place, that is fine.


I assume you just placed the json parser unit into your project folder? That would be fine too.

I didn't add any paths manually, but there is a total of three paths listed which were added automatically by using the "Add" function of the project inspector. All three paths point to sub-folders in my project folder. All files in these paths are units, the separate folders are just to keep things organised.
I don't have any paths pointing to Lazarus or FPC sources, no.

Yes, the json parser unit is in the project folder.

However, now it gets interesting (maybe, if I am not mistaken).

LCL (and dependencies) would not likely be rebuild.
Unless you change the "configure rebuild Lazarus" settings, or you have "additions and overrides" affecting packages in your project options (see that tab in project opts).

You can also watch the messages windows: Does it say anything about "compiling package ..."? Or does it only output stuff about compiling your project?

Because, if LCL and dependencies is not recompiled, and if you have no other packages.... And if I am right, then there should be only one instance of fpc and one of ppc. (Someone of the fpc team might be able to confirm that).

Of course if you don't notice that that ppc___ didn't stop, and you compile again, then you would have two.

But I understand that a single compilation (one time pressing F9, or one time choosing "build") leads to several ppc___.

I don't have anything in "Additions and Overrides" right now, although I did use it a few times to compile the project with qt5 in order to test it against issues I was encountering with gtk2.

I'm not entirely sure about "Configure "Build Lazarus"". I've had to rebuild it once or twice for whatever reason, but that might even date back to a separate installation of an older version of Lazarus and I don't think I've used anything beside the default settings. So probably not relevant here.

A quick Ctrl+F through Messages after a successful run does not show any instance of a "compiling package" message.


I just got word from a member of the fpc team. If really only your project is buid, then there should be only one fpc and one ppc process (and both are single threaded).

That is, if the IDE only starts one compilation. Which in this case it should.

You don't have any custom settings in "complier commands" of the project settings?


No, I don't have any custom settings in "Compiler Commands".

What I've observed so far suggests that a single run/compile doesn't lead to several instances of fpc/ppcx64 by itself, but that the instances created by a single run don't end after the run, even when the app is closed, probably because they are still stuck in the endless loop that causes the initial issue for which I opened this forum thread. So they end up "orphaned", and new instances are created on the next run, leading to a pile-up over time.

The problem with the thread locks is:
- indeed you can compile lazarus as a whole divided over several threads, I believe it is even the default.. And multiple ppc's as a result.
- but packages can rely on shared units and resources, like sysutils, objpas, etc, etc, etc, even ppu's.
- and such constructs can subsequently cause a thread deadlock issue.
It is not something that happens often but it does happen every now and then, that's why I turned that off.
Building Lazarus with several threads does not always work very well. If it works, it works much faster, though. 9 out of 10 cases or better.
It is a bad luck = bad lock issue. Building FPC itself does not suffer from that issue.
It is darn hard to debug, so I chose the easy way out.
You can check that this is indeed the case in process windows on several platforms like windows and linux.

I assume this isn't entirely relevant here since I'm not building Lazarus or any packages, but it might be worth a try: how can I turn off multi-threading?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #18 on: November 13, 2024, 03:59:43 pm »
Well the IDE is always multi threaded. FPC is never.

To reduce the IDE thread usage in respect to managing multiple compilers: Tools > Options > Environment > Messages Window : "Maximum parallel processes..."
Set it to: 1

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #19 on: November 14, 2024, 02:18:25 pm »
Well the IDE is always multi threaded. FPC is never.

To reduce the IDE thread usage in respect to managing multiple compilers: Tools > Options > Environment > Messages Window : "Maximum parallel processes..."
Set it to: 1


Thank you. I've given it a try, but no luck. The issue occurred all the same.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #20 on: November 14, 2024, 02:36:45 pm »
My guess, the only way to get answers on that is to debug fpc itself. Build fpc with debug info, wait for it to hang, attach a debugger, and step until finding where it loops, and what condition causes it.

Alternatively, if you can 100% reproduce it, then start reducing the code for reproduction and report it with a small example.
Such an example should ideally work from command line (not need the IDE).
- Have a folder, with only the sources. (no lib/unit folder/ no ppu or o files). Additionally folders with pre-compiled packages, if this can not be avoided.
- First compile will likely work.
- Then I guess one source needs changes, and next compile will hang.

But finding such an example will be a lot of work, and possible also depend on luck. (or on debugging fpc first).

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #21 on: November 17, 2024, 08:49:25 pm »
My guess, the only way to get answers on that is to debug fpc itself. Build fpc with debug info, wait for it to hang, attach a debugger, and step until finding where it loops, and what condition causes it.

Alternatively, if you can 100% reproduce it, then start reducing the code for reproduction and report it with a small example.
Such an example should ideally work from command line (not need the IDE).
- Have a folder, with only the sources. (no lib/unit folder/ no ppu or o files). Additionally folders with pre-compiled packages, if this can not be avoided.
- First compile will likely work.
- Then I guess one source needs changes, and next compile will hang.

But finding such an example will be a lot of work, and possible also depend on luck. (or on debugging fpc first).


Reproducing the issue is not an option right now, I haven't the slightest inkling what causes it.

As for trying to debug the fpc, whether that is doable depends on how complicated and long-winded it would be to do. Is there a guide somewhere?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #22 on: November 17, 2024, 09:45:31 pm »
As for trying to debug the fpc, whether that is doable depends on how complicated and long-winded it would be to do. Is there a guide somewhere?

Not that I am aware off.

Since you can't reproduce, it would be a case of debugging after it started to happen. Since FPC runs forever, it remains happening.

Therefore
- build an fpc with debug info. (ppc... must have debug info). Can be done with FpcUpDeluxe.
- Wait till it happens
- Start an extra IDE (must be set to allow multiple IDE to run, and not re-use the already running)
- From that 2nd IDE attach to ppc....
- Hit Pause
- Under Windows switch to the main thread (Windows itself adds a thread when you pause an app)
- Record the stack
- Search the stack -or- Step / step out  until you find some code with a loop that keeps looping.
- Record as many vars as you can. Including stepping into some of the methods...

Hope someone with the FPC team can tell you what the code in question does, and if the value from the variables are meaningful.

It's the kind of desperate move. No promise ... Just a last straw.

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #23 on: November 20, 2024, 09:41:58 am »
Not that I am aware off.

Since you can't reproduce, it would be a case of debugging after it started to happen. Since FPC runs forever, it remains happening.

Therefore
- build an fpc with debug info. (ppc... must have debug info). Can be done with FpcUpDeluxe.
- Wait till it happens
- Start an extra IDE (must be set to allow multiple IDE to run, and not re-use the already running)
- From that 2nd IDE attach to ppc....
- Hit Pause
- Under Windows switch to the main thread (Windows itself adds a thread when you pause an app)
- Record the stack
- Search the stack -or- Step / step out  until you find some code with a loop that keeps looping.
- Record as many vars as you can. Including stepping into some of the methods...

Hope someone with the FPC team can tell you what the code in question does, and if the value from the variables are meaningful.

It's the kind of desperate move. No promise ... Just a last straw.

Thank you for the step-by-step. I've given it a quick try, but when trying to attach to the ppc process from the second IDE, I get the message "Debugger Error: Error starting process in debugger." Is that something I can fix? Or did I do something wrong in following the steps?

Just to clarify: in order to install the fpc with debug info through FpcUpDeluxe, I went to "Setup+", and toggled "Debug" for FPC in "Options Override" - and based on one of your own posts elsewhere I also added "-gw" in the "FPC Options" edit. Was that correct?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #24 on: November 20, 2024, 10:16:54 am »
Sounds all correct.

Not related to your error, you may want to open one of the ppc__.lpi e.g. "ppcppc64.lpi" from the fpc source folder => then the IDE will find the sources. But that doesn't affect the attach itself.
(Well if you had a library project open then that could have caused the error)
=> you can try to add the ppc___.exe (with full path) in "Run Params" as "Host application".

What OS are you on?
Which version of Lazarus?
Which debugger do you use (fpdebug or gdb)? (if Mac then lldb + fpdebug?)



If you watch the ppc in the task list (windows task explorer - details pane / or "top" on Linux) => They keep the same PID?

Just to be sure it is one long running process. and not something restarting ppc over and over.

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #25 on: November 23, 2024, 06:06:54 pm »
Ok, thank you. I'll give that a try next time the issue occurs.

My OS is Ubuntu 24.04.1 LTS.
My version of Lazarus is 3.99.
I'm using the default fpdebug as my debugger.

As far as I can tell, the PID of each ppc process stays the same.

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #26 on: November 30, 2024, 03:11:01 pm »
Sounds all correct.

Not related to your error, you may want to open one of the ppc__.lpi e.g. "ppcppc64.lpi" from the fpc source folder => then the IDE will find the sources. But that doesn't affect the attach itself.
(Well if you had a library project open then that could have caused the error)
=> you can try to add the ppc___.exe (with full path) in "Run Params" as "Host application".

Ok, so I've given it another try after following everything you suggest here, but I'm still getting a Debugger Error when trying to attach: "Internal error: No filename given to execute".

I've tried every instance of ppc I've had running, but same result each time.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #27 on: November 30, 2024, 05:46:41 pm »
Ok, so I've given it another try after following everything you suggest here, but I'm still getting a Debugger Error when trying to attach: "Internal error: No filename given to execute".

I've tried every instance of ppc I've had running, but same result each time.

It depends on the project that you have open.

The simplest workaround should be to go to "run param" and set a "Host application". Pointing to any exe should do, but ideally to the ppc.

Nel_Fie

  • Jr. Member
  • **
  • Posts: 52
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #28 on: December 01, 2024, 09:49:48 am »
It depends on the project that you have open.

The simplest workaround should be to go to "run param" and set a "Host application". Pointing to any exe should do, but ideally to the ppc.

Yes, you've mentioned that in your previous post, but I might not have executed the idea as intended. Which instance of Lazarus am I supposed to do this with? The first one, which runs the project and will actually cause the issue itself; or the second instance of Lazarus, which I'm using to debug the issue?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10553
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus/FPC semi-randomly gets stuck while compiling
« Reply #29 on: December 01, 2024, 12:22:41 pm »
The one in which you are selecting "attach" from the menu.

It is a bug in the debugger. => normally when you start the debugger you need an executable that the debugger will launch.
But when you attach, then you don't, since the executable is already running.
The check is incorrectly performed.

So until that is fixed, the workaround is, to give the debugger an exe (and ideally the correct exe / though it probably works with any exe). It will not be launched, but it will get through the check.

 

TinyPortal © 2005-2018