FpDebug question(s)

[440bx]

Hello,

For the time being, these questions are mostly a matter of curiousity.

First one is, can FpDebug dynamically load an external file (.dbg) when execution goes into dll code ?  e.g, a Windows program goes into user32, ntdll or some other system dll.

In MS Visual Studio, when execution goes into a system dll, VS uses the debug information stored in the dll to download the appropriate symbols (or load them from the local cache), how much work and how complicated would it be to enable Lazarus to load external .dbg files on demand based on the dll's stored debug information ? presume that there is a .dbg file whose name is the guid found in the dll's debug descriptor (to enable unambiguous identification.)

Thank you for your help.

[Martin_fr]

First of all => that is talking about dbg files with DWARF info. Since FpDebug only loads dwarf. (Happy to accept merge request for other formats, but currently not on my list).

If they are on the disk, they should be found. Though I haven't tested that. I know FpDebug loads the dwarf if it is inside the library itself. It uses the same loading code, and that code supports dbg files (on the disk).

If they point elsewhere, I would probably deal with that via a callback hook to the IDE, and let the IDE download them (if the user enabled that option). Shouldn't be to complicated.

The relevant code in is the FpImageReader[Elf|Pe]. (IIRC).


System dll on windows usually don't have dwarf info. They use a different format. (afaik)

[440bx]

First of all => that is talking about dbg files with DWARF info. Since FpDebug only loads dwarf. (Happy to accept merge request for other formats, but currently not on my list).

Yes.  I am also definitely limiting the considerations to DWARF info.  Implementing PDB processing requires too many dependencies on proprietary MS technologies which, I believe would not be beneficial to Lazarus.

If they are on the disk, they should be found. Though I haven't tested that. I know FpDebug loads the dwarf if it is inside the library itself. It uses the same loading code, and that code supports dbg files (on the disk).

I'd like to test FpDebug's ability to load an external disk based .dbg file (I don't care about other possible locations.)  Do you have any suggestions as to how  I should go about doing that ?  thank you in advance. 

The relevant code in is the FpImageReader[Elf|Pe]. (IIRC).

Good to know.  I'll have look at it.  Thank you.

System dll on windows usually don't have dwarf info. They use a different format. (afaik)

Yes, they use the MS proprietary PDB format but, there is nothing that prevents writing a program to create DWARF symbols for them  

[Martin_fr]

Well FpDebug expects a "gnu_debuglink" section that has the link to the filename. TDbgImageReader.LoadGnuDebugLink
Fpc created that if you compile something with external debug info. (it has a crc to check that it is up to date)

FpDebug looks in the path of the exe, and in <exepath>/.debug/


I haven't investigated what .so files off the linux kernel contain. I know gdb has some download ability for them. But I know no details about that.
**If** the are the same format, and if you have them

TElfDbgSource.Create  calls the LoadGnuDebugLink => just hardcode name and location

[440bx]

Well FpDebug expects a "gnu_debuglink" section that has the link to the filename. TDbgImageReader.LoadGnuDebugLink
Fpc created that if you compile something with external debug info. (it has a crc to check that it is up to date)

FpDebug looks in the path of the exe, and in <exepath>/.debug/

Very useful information.  Thank you Martin.

[Martin_fr]

You may find this of interest. https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/41033#note_2141894188

[440bx]

Thank you Martin.

Yes, I'm definitely interested in anything that is related to FpDebug's abilities and behavior.

[Red_prig]

Martin_fr, I am wondering if it is possible to add a modified fpdebug, for example via an installable package? I need some advanced features on the debugger side, can I write a plugin/package for this?

[Martin_fr]

You can register new backends via a package. (That would be a modified LazDebuggerFp, which could then use a modified FpDebug).

The existing frontend will use the Intf to communicate with them (so the actions/data) is fixed there. And keep in mind, that intf will change)
But you can register additional stuff via IDE intf, and talk directly to your package.

[PascalDragon]

First of all => that is talking about dbg files with DWARF info. Since FpDebug only loads dwarf. (Happy to accept merge request for other formats, but currently not on my list).

Yes.  I am also definitely limiting the considerations to DWARF info.  Implementing PDB processing requires too many dependencies on proprietary MS technologies which, I believe would not be beneficial to Lazarus.

LLDB supports the PDB format (see here).

Also Microsoft published information about the format to help LLVM further.

LLVM documents it also here.

So it isn't that proprietary anymore.

[Martin_fr]

System dll on windows usually don't have dwarf info. They use a different format. (afaik)

Yes, they use the MS proprietary PDB format but, there is nothing that prevents writing a program to create DWARF symbols for them

I wouldn't mind FpDebug to be extended to read it directly. If someone wants to do that.

Assuming that there are no licensing issues, but if lldb does then....

[440bx]

@PascalDragon,

I knew about the LLVM folks having fully reversed engineered the PDB format as well as MS' contribution (which doesn't compile!), OTH, the last time I looked at their stuff was a few years ago (more than 4) and, at least at the time, there was no _published_ documentation from LLVM about the PDB format (and following their code was  no picnic!.)

I see now that the documentation is much better and, from the link you posted, there is code that seems reasonably usable to understand the format without incurring a very steep learning curve.

Thank you for the update.  It's good to know there have been improvements in that area.

I wouldn't mind FpDebug to be extended to read it directly. If someone wants to do that.

Assuming that there are no licensing issues, but if lldb does then....

Layman's opinion:  given LLVM's ability to use the format (which they reversed engineered with only some/partial help from MS), it seems MS is "ok" with third parties using the format (but, it is still suspicious they don't document it.)

I've been "looking" at the PDB format for a long time, since Undocumented Windows 2000 by Sven Schrieber.  Since MS doesn't document it, any changes, which there have been a great many through the years, could be problematic for any program directly reading the format.

Personally, I'm leaning towards using the "cv2pdb" approach which uses MS provided facilities (API's) to read the PDB and convert the information to DWARF.  I believe that is a safer and more stable approach in the long run.  The downside is: an external ".dbg" must be created and Lazarus be given the ability to find it so it can use it.

That said, I certainly would not have anything against someone creating a direct PDB reader in Pascal with could potentially be included into Lazarus but, as previously stated, I think it is risky in the long run.  I can't help feeling uneasy about it.


Somewhat related observation about the PDB symbols.  I routinely dis-assemble MS executables (mostly Windows system dlls) and, one of the things that is noticeable is that the debug information they publish has been "fiddled" with to some extent.  This lowers the value of the PDB information they make available. 

It would be _very_ nice to augment/correct the PDB information.  I think that, when it comes to creating debugging symbols, the DWARF specification is a more reasonable/safer avenue.  That's the objective that has captured my attention for the time being.

[Alexander (Rouse_) Bagel]

That said, I certainly would not have anything against someone creating a direct PDB reader in Pascal with could potentially be included into Lazarus but, as previously stated, I think it is risky in the long run.  I can't help feeling uneasy about it.

I have plans to develop a PDB-Reader in RAW format for my ProcessMemoryMap utility.
https://github.com/AlexanderBagel/ProcessMemoryMap
Now it works only with debug symbols provided by WinAPI (SymInitialize, SymGetSymFromAddr, etc...), with Delphi MAP files and DWARF(1-4, partially 5).
(Sorry, the documentation for the utility is still in Russian, but there is nothing important there)
But now work on this utility is on hold as I need to finish work on CPU-View and assembler/disassembler framework (it is needed for my own debugging kernel).
As soon as I finish these projects, I will tackle the PDB immediately with the possibility of connecting to Lazarus in mind.

[440bx]

Now it works only with debug symbols provided by WinAPI (SymInitialize, SymGetSymFromAddr, etc...), with Delphi MAP files and DWARF(1-4, partially 5).

I've "played" with the dbghelp functions used to extract information from a PDB.  Mostly proof of concept stuff, nothing hard core.

Because of your comment about developing a PDB reader, I have a question for you:  do you know if there is information a direct PDB reader can obtain that is not available using the dbghelp functions ?  (I ask because, if no additional info can be had reading the PDB directly, it seems there is no net advantage to implementing one.)

That said, if you developed one, I'd be among the firsts to have a good look at it.

BTW, I had already seen your ProcessMemoryMap and one thing that caught my attention was your use of distorm to disassemble code.  I've used distorm and like it but, I've switched to Zydis because it supports practically all intel instructions and it is well tested (and used everyday as it is the disassembler used by the open source x32/64dbg)

[Alexander (Rouse_) Bagel]

do you know if there is information a direct PDB reader can obtain that is not available using the dbghelp functions ?  (I ask because, if no additional info can be had reading the PDB directly, it seems there is no net advantage to implementing one.)

I haven't really delved into the PDB issue yet, so I can't answer what exactly might be  not available.
But I have superficially studied this project, on the basis of which I want to make my own implementation of PDB format reading.
https://github.com/MolecularMatters/raw_pdb/tree/main/src

As for distorm - its use is a forced measure, in the future it will be replaced by its own disassembler engine (its previous implementation was limited in capabilities), as I need to perform assembly and then modify the code in a remote process.