Debugging on Mac super slow to switch units

[Grahame Grieve]

I'm debugging using the v4 RC release (Lazarus 4.0RC1 (rev Unknown) FPC 3.2.2 aarch64-darwin-cocoa) on Apple Silicon (M1 Max, 64GB, Sequoia 15.2).

While debugging my application - a multi-threaded server that's a pretty big compile - if I switch units in the source display, Lazarus can hang for 1-2 minutes each time. Sometimes it will, sometimes it won't. This makes debugging pretty hard - does anyone have any idea what's happening?

[Zvoni]

How many threads? Threads running only on the CPU or also on the GPU?
If only CPU, remember that the M1 Max has 8/10 cores, so maybe there?

[Martin_fr]

Open Menu: View > IDE internals > Debug Output.

Then when you switch units, check if there is any output added to that Window.


Also check if disabling (power button off) for stack or watches makes a different.
In order to test that, first open the "Debug > History" Window and switch its power button to off.

There was an issue where the stack (if it showed lots of entries) would be slow to resolve the files listed in the stack (even if the stack already was shown and not changed). Though that was fixed (at least as far as I was able to test) before 4.0RC


Also, about "sometimes": Check if that "sometimes" is the first time you switch to that unit, during the debug session.
When you switch to a unit, and that unit had not yet been "switched to" since the debugger was started  (including already switched to at the time the debugger started), then the IDE loads the "blue dots". If your app is massive then it needs to go through a lot of debug-info (IIRC on Mac, it needs to locate all involved .o files, and check each of them).

Mind that foo.o can contain lines for bar.o, if foo specializes a generic that is from bar. Since the debugger can not know if that happened during compilation, it must check all .o files.
On other OS the entire info is usually in a single file (either the exe, or a dbg file).

It's been a while since I looked at Mac details, but afaik on Mac the info is spread across lots of individual files. So if you have 1000 files that all have debug info, then the debugger needs to open them all.
Though (if memory serves), it should read them once, and keep line info in memory. But not sure....

Mind that the debugger does need some amount of memory. So if you are low on memory, it likely needs to swap a lot.


Also, please confirm you are using "LLDB with FpDebug"

[Grahame Grieve]

Confirm:

* nothing new in the debug output window
* power button off doesn't make any difference
* using LLDB with FpDebug
* not short on memory (have oodles)
* it's only the first time I switch to a unit, and sometimes only once during the debugging session, and sometimes again for a new unit

There is a lot of .o files, that's true - 100s of them. It's hard to count though.
is there anyway that the IDE can load this faster? A 1+ minute hang isn't something that happens with any other IDE.
And if it can't, can it at least not hang hard while it's doing this?
 

[Martin_fr]

It will likely be needed to take some timings (add some writeln).
The loading of the line info involves a lot of parts of the IDE. So without knowing where the time goes...

On 2nd thought, that line info normally gets accessed right when you start debugging (at least if you have at least one breakpoint, maybe even without that). Though it gets processed slightly different. (And that part was recently changed, and made faster / I have to check if that went into 4.0).

So, if you set a breakpoint anywhere, how long does it take to start the app in the debugger?


For the rest, I have to go through the code and find points where I can add writeln to measure timings.


Also looking for further pointers.
Is it always the same units for witch "switching to" is slow? 
Do they have particularly high amounts of lines? (code and/or comment/empty) ?
Do they define generics, that are heavily used (specialized often) throughout other units?
Do they have include files / Are they include files?

Do those slow units show the blue dots correct? (on all lines with code, and only on lines with code?)
(that may include begin/end lines)

Additionally, please run your IDE from the console, and watch for any output to the console while the IDE is hanging.

[Martin_fr]

I'm debugging using the v4 RC release (Lazarus 4.0RC1 (rev Unknown) FPC 3.2.2 aarch64-darwin-cocoa) on Apple Silicon (M1 Max, 64GB, Sequoia 15.2).

Did you previously use 3.n? With the same compiler, and the same project? Was that significantly faster?

Is your compiler build with debug info (can you step into the RTL)?

[Martin_fr]

I prepared some time measurements.

https://gitlab.com/martin_frb/lazarus/-/compare/main...b5e1f013375e1b8bda62d3f869f89e1b557f2965?from_project_id=28419588

If you can either apply it as a patch, or download this version (based on current 4.99 / main branch).

Then run the IDE from a console and start it with
   lazarus --debug-log=somefilename.txt

After experiencing the slow down:
- close the IDE (so the file is flushed)
- upload the file

If your app has data you do not wish to become public, check the log that it does not accidentally contain any such data. (e.g. function/variable names).

[Martin_fr]

Also please check if you are using Dwarf-2 or Dwarf-3.

Does it make a difference if you change that?

If the files that slow down are part of a package, then check the settings for that package.