[Solved] gdb crashes when hovering over an array... :-O

[Hoch]

I'm working on a large project with Lazarus and gdb. Both have been working fine for months.

Yesterday, I was making some changes and suddenly, gdb started crashing when I hover over an array... 

I append several screenshots of a little program I made to confirm that it happens with any project. When I hover over `bar[1] := 1` to see its contents, I get the error shown in the screenshot and gdb exits with a fatal error.

I reinstalled gdb and fpc. compiled with 'cleanup and build', restarted de debugger, but nothing helps.

I'm using Lazarus 2.2.0 compiled with QT, with fpc 3.2.2 on an Arm sbc. The os is Manjaro (linux) with kernel 6.2.

It seems that something changed in gdb. How can I reset it?

Thank you and greetings, 

[Fred vS]

Hello.

Not sure if it can help but I get the nearly same surprise after a upgrade of XUbuntu 22.10.

Code: Bash  [Select][+][-]

1. sudo apt get upgrade

GDB did not work correctly anymore.
The XUbuntu version of gdb 12.1 has problems.

The solution was to download the gdb source from here: http://mirrors.kernel.org/sourceware/gdb/releases/
And compile + install with this:
./configure, make and sudo make install.

With this gdb is working again.

I did try with last version gdb 13.1  (compile + install) and gdb is working perfectly too.

[Martin_fr]

It's hard to say what changed. Could even be the content of the array.

Before we go into depth... Have you tried FpDebug?
You can set it under menu: Tools > Options > Debugger > Debugger Backend.
It might be in the list (dropdown in toolbar on top of page) already, if not "add" button, and then chose under "debugger type".

On the issue itself.

You (probably) did not change: Lazarus or Fpc.

I don't know if your gdb got recently updated.
It could also be the linker.
Or any library that is linked into your app (gtk, clib, .....)

Since you were able to extract it into a small sample app, it is not likely related to your surrounding code.
And a static array of integer, is normally not something that would crash gdb.

There was a recent issue that may be related: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40148
The issue appears (one first glance) to be caused by a specific version of glib.


If you must use gdb (instead of FpDebug), the first suggestion would be to try and get a different version of gdb. (even an older could help)

You can also try to check/enable DisableLoadingSymbolsForLibrary (in your first image). Though it's a long shot.

[Hoch]

Hi Fred vS and Martin_fr,

Fred vS, thanks for your fast answer.

No I haven't upgraded my system, nor Lazarus, fpc, etc.

It happened suddenly when I was coding.

I was using gdb 13.1 without issues.

Martin_fr, thanks for your support all those years! 

When I first installed Lazarus on this AARch64 system (a Radxa Zero) with Manjaro Linux, I tried to use FPDebug, but found issues that I did not have in my Intel laptop (I can't remember which ones, it was several months ago).

So I had to use gdb.

Maybe those issues are fixed now.

I tried to install FPDebug today but I got the following error: 'Error starting process in debugger'. I appended a screenshot of the error and the settings.

Maybe some package is missing? I appended a screenshot of the installed packages in the first message. I usually compile Lazarus with minimum packages.

Thanks! 

[Martin_fr]

I didn't know you where on arm (AARch64).
FpDebug only runs on Intel (and AMD). So no, it wont work for you. Unless - maybe -you compile for Intel, and then run in emulation.

If nothing helps, there is an lldb wrapper for the IDE too. But that is very spartan. Really not recommended (it exists as base for a fpdebug extension). Package LazDebuggerLldb.

What you can try...  (but its just trial and error)

- do a clean build / manually remove all .ppu .o .dbg files from your project folder
  this includes any folder listed in the "path" tab of the "project options", and any subfolder there of.
  (Unless you put some files there for 3rd party stuff, and that came without source.)

- Don't use external debug info
- Try different debug formats: dwarf2 (with/without sets) vs dwarf3.  (or even dwarf 4, as custom option -gw4 )
- "stabs" may no longer be supported, but try it...
- not sure if available in fpc 3.2.2, try the custom option:  --godwarfcpp

- Try "strip  --only-keep-debug yourapp"
- Try fpc 3.2.3 (not expecting much of it) or 3.3.1 (though that also falls under, "if desperate")

Your app also includes debug info for the LCL, and other packages. Rebuild those either without, or with the same info as your use in your app.
This is controlled in menu: Options > Configure build Lazarus
There is a field custom options. IIRC for no debug: -g-
Or add in project options "additions and overrides", target *

[Hoch]

Hi Martin_fr,

Thanks for all the wise suggestions. I tried everything in the list, but nothing worked for this strange gdb crash.

I also tried to revert to a previous version of gdb from the Arch installer (Pamac, GUI of Pacman command) but couldn't do it without unistalling Lazarus first because gdb is marked as a dependecy of Lazarus.

Finally, I managed to revert to gdb version 12.1.

Those are the steps for Arch distributions:

1. Renamed the /home/user/.Lazarus preferences folder to .Lazarus.bak,

2. Uninstalled Lazarus with Pamac,

3. Uninstalled gdb 13.1 and gdb-common 13.1 with Pamac,

4. reverted to a previous version of gdb, version 12.1, that was stored in the Pacman cache, using this commands:
   $ sudo pacman -U file:///var/cache/pacman/pkg/gdb-common-12.1-2-aarch64.pkg.tar.xz
   $ sudo pacman -U file:///var/cache/pacman/pkg/gdb-12.1-2-aarch64.pkg.tar.xz
   
5. In Pamac preferences --> Advanced --> Ignore list, I blacklisted gdb and gdb-common, so it was not upgraded to 13.1 when reinstalling Lazarus,

6. Installed Lazarus QT with Pamac,

7. Renamed the /home/user/.Lazarus.bak to .Lazarus,

8. Launched Lazarus...

Et voilà! All back to normal... No more crashes hovering arrays.

I don't know why gdb 13.1 started to crash. I was using it since last month, when I received the upgrade.

Hope this helps other Arch and Manjaro Linux users, on ARM Aarch64.

Thanks for your fast an thorough help and keep up the good work! 

[Martin_fr]

Ah, one more thing comes to mind...

IIRC gdb can cache some stuff on the disk. I don't have much background on that.
But if something ended up wrong....

Check when you start gdb, if it prints something like "path for the index cache"
Or just try the usual candidates, such as ~/.gdb

[Hoch]

Hi Martin_fr,

Yes, gdb caches, this makes sense and could be the source of trouble. Maybe my gdb cache was corrupted and had to be cleared...?

I have no experience with gdb. So I asked the phind.com AI how to clear the gdb cache...

GDB caches data exchanged between the debugger and a target. The cache is associated with the address space of the inferior (the process being debugged). Caching generally improves performance in remote debugging by reducing the overhead of the remote protocol by bundling memory reads and writes into large chunks. However, caching everything would lead to incorrect results, since GDB does not necessarily know anything about volatile values, memory-mapped I/O addresses, etc. Furthermore, in non-stop mode, memory can be changed while a gdb command is executing. Therefore, by default, GDB only caches data known to be on the stack or in the code segment.

To explicitly mark other regions of memory as cacheable, we can use Memory Region Attributes. Additionally, we can enable or disable caching of stack accesses or code segment accesses using the commands set can-use-cache on or set can-use-cache off. By default, caching of stack and code segment accesses is enabled.

To clear the cache, we can use the command flush data-cache. This command flushes the contents (if any) of the data cache.

It is important to note that clearing the cache may affect the performance of the debugger, and may cause the debugger to re-read data from the target, which can slow down the debugging process . Therefore, clearing the cache should only be done when necessary.

In summary, to clear GDB debugger caches, we can use the command flush data-cache. However, clearing the cache may affect the performance of the debugger, and may cause the debugger to re-read data from the target, which can slow down the debugging process. Therefore, clearing the cache should only be done when necessary.

So to clear the memory cache, on the command line, run gdb and then 'flush data-cache' or 'maintenance flush register-cache'. You can also disable the cache with set can-use-cache off.

Also there is the index cache you mention, that is stored on disk. To show the path to the index cache run gdb and then type 'show index-cache directory'. Mine is located in /home/my_user/.caches/gdb. To clear it, just delete the contents of this gdb folder.

Thanks Martin_fr!