Recent

Author Topic: Lazarus, FpDebug and GDB  (Read 1786 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus, FpDebug and GDB
« Reply #15 on: November 29, 2022, 04:30:34 pm »
Well technically there is a lot in your code, that could eventually benefit FpDebug.

Stack unwinding partly is a guessing game. Debug info may contain info, that is sufficient, and allows to get the right result.
But not all code has debug info, and some callers may not have that.

If callers don't have it, it relies all on certain assumptions (stack base pointer / rbp).

What would be good would be detecting prologue and epilogue code (not just fpc, but kernel and libs ...). Because within that, RBP is not reliable.
Other ideas (though really lot of work) would be to find a path to the "ret" statement, and track the stackpointer.

Prologue and epilogue  is something you might need yourself. Because knowing how the stackframe is created, is a first clue to local vars. (I.e. param in register is saved to a location on the stack / though that depends how optimized the code is, and if it goes to the stack at all).

440bx

  • Hero Member
  • *****
  • Posts: 3946
Re: Lazarus, FpDebug and GDB
« Reply #16 on: November 29, 2022, 04:53:14 pm »
Well technically there is a lot in your code, that could eventually benefit FpDebug.
definitely.  the more comprehensive and accurate the analysis of the code, the easier it becomes to manage and debug.

What would be good would be detecting prologue and epilogue code (not just fpc, but kernel and libs ...). Because within that, RBP is not reliable.
In 64bit, the exceptions table are quite useful for that.  Unfortunately, that's only available for 64 bit executables.

Other ideas (though really lot of work) would be to find a path to the "ret" statement, and track the stackpointer.
IDA Pro tries to do that and in most cases it does well but, there are cases it cannot handle.  A lot of analysis is required to get it right.

Prologue and epilogue  is something you might need yourself.
Most definitely.  That's one the reasons the PE dump program I wrote is as detailed as it can get when showing the exceptions tables.

For now, back to square one, get a good understanding of DWARF  :D
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus, FpDebug and GDB
« Reply #17 on: November 29, 2022, 05:02:03 pm »
Well a point on Dwarf, you might find "good to know":

There is a need to modify the exe (elf or pe file).
Even external debug info, needs a section added to the existing exe. This section will say: there is external debug info, and it has the following checksum.
At least gdb relies on that (and fpdebug too)

Of course if you don't want the need of modifying the exe, you can modify the debugger to find the external info without that section. (of course if you can modify the debugger)

No idea what PDB does.

440bx

  • Hero Member
  • *****
  • Posts: 3946
Re: Lazarus, FpDebug and GDB
« Reply #18 on: November 29, 2022, 06:05:05 pm »
Well a point on Dwarf, you might find "good to know":

There is a need to modify the exe (elf or pe file).
Why is that ? I don't see the reason why there is a need to modify the exe... enlighten me please.

This section will say: there is external debug info, and it has the following checksum.
At least gdb relies on that (and fpdebug too)
I can see the need when the symbolic information is kept outside the exe but, I don't see it if the symbolic info is in the exe itself.


No idea what PDB does.
In the case of a PDB, which is always external, the debug directory contains the name and location (if local) of the of the PDB file and the PDB file's associated GUID to ensure the PDB is version-wise matched to the exe.  For instance, the following is the debug directory for a version of conhost.exe

Code: Text  [Select][+][-]
  1.  1-4009.ed90            - DEBUG DIRECTORY -
  2.  
  3.  1-4009.ed90                 DEBUG DESCRIPTOR : 1
  4.  
  5.  1-4009.ed90  9.db90  [4]              Characteristics :         0
  6.  
  7.  1-4009.ed94  9.db94  [4]              Time Date Stamp : fd66.3755    (2104/09/20 6:14:45)
  8.  
  9.  1-4009.ed98  9.db98  [2]                Major Version :         0
  10.  1-4009.ed9a  9.db9a  [2]                Minor Version :         0
  11.  
  12.  1-4009.ed9c  9.db9c  [4]                         Type :         2
  13.  1-4009.ed9c +                     DEBUG_TYPE_CODEVIEW
  14.  
  15.  1-4009.eda0  9.dba0  [4]                 Size of data :        24    (36)
  16.  
  17.  1-4009.eda4  9.dba4  [4]    Address (rva) of raw data :    a.4fc0    [Va: 1-400a.4fc0] [FO: a.3dc0] [  .rdata]
  18.  1-4009.eda8  9.dba8  [4]   (File) Pointer to raw data :    a.3dc0
  19.  
  20.  1-400a.4fc0                 DEBUG_TYPE_CODEVIEW
  21.  
  22.  1-400a.4fc0                 a.3dc0  [ 4]         Signature :  RSDS
  23.  
  24.  1-400a.4fc4                 a.3dc4  [16]              GUID :  45421771a8142d0cf308c2f6fdabe239
  25.  1-400a.4fc4 +                                                {71174245-14a8-0c2d-f308-c2f6fdabe239}
  26.  
  27.  1-400a.4fd4                 a.3dd4  [ 4]         Iteration :  1
  28.  
  29.  1-400a.4fd8                 a.3dd8  [12]      Symbols file :  conhost.pdb
  30.  
  31.  
  32.  1-4009.edac                 DEBUG DESCRIPTOR : 2
  33.  
  34.  1-4009.edac  9.dbac  [4]              Characteristics :         0
  35.  
  36.  1-4009.edb0  9.dbb0  [4]              Time Date Stamp : fd66.3755    (2104/09/20 6:14:45)
  37.  

(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus, FpDebug and GDB
« Reply #19 on: November 29, 2022, 08:00:51 pm »
Well a point on Dwarf, you might find "good to know":

There is a need to modify the exe (elf or pe file).
Why is that ? I don't see the reason why there is a need to modify the exe... enlighten me please.

Afaik just convention.

Now I can't say I checked gdb sources.... So not sure what they allow. But as far as I know, gdb checks the exe, and it will only look for the "dbg" file, if it finds a ".gnu_debuglink" ( <= google this name) section in the exe.

It was therefore implemented the same way in FpDebug.  (search for "ReadGnuDebugLinkSection")
Yet in FpDebug we can of course add other ways.

The CRC is used to protect against outdate files.
The CRC is stored inside the main exe (it that section). And it is the CRC of the external file.
IMHO, it would have been better to reverse that. To store the CRC of the exe (or at least of some of the significant parts of it, to allow for linking ....) in the ext .dbg file.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus, FpDebug and GDB
« Reply #20 on: November 29, 2022, 09:00:24 pm »
One other think I thought about pointing out (kind of an idea, almost radical)...

If I understand you correct, you start out on an unknown exe. No source, no debug info.
You disassemble, you analyse, and you take guesses.
Then you have like
- this proc might be...
- this memory might be variable typed ....
And then you add that as dwarf, and let the user run it in a debugger, and the user can then inspect the data, and amend the guesses.

Now your plan is to write debug info in an existing format, and use an existing debugger.

Within FpDebug you have access to controlling an exe in memory (lots of it work without debug info, some wants debug info ... but that does not need to come from a file. (And if/where FpDebug does not have proper separation, that could/should probably be fixed anyhow).

So instead you could store your guesses in memory, and debug in a special debugger, that accesses your debug info.

You may even get along without any debug info to FpDebug. (Unless you want to have line based stepping, but that only works if you grouped asm blocks into lines).

For watches, you use FpDebug to just read the memory. And then you use your guesses to display the memory content.

It will be some work... of course...
And the above needs to be double checked.

440bx

  • Hero Member
  • *****
  • Posts: 3946
Re: Lazarus, FpDebug and GDB
« Reply #21 on: November 29, 2022, 10:52:27 pm »
".gnu_debuglink" ( <= google this name) section in the exe.
Yes, I had seen that but, only paid enough attention to it to determine FPC does not emit one.

It was therefore implemented the same way in FpDebug.  (search for "ReadGnuDebugLinkSection")
if I am understanding you correctly, it seems that FpDebug's ultimate goal is not just being a debugger for FPC but also one that could be used for any other GNU language.  Is my impression correct ?



If I understand you correct, you start out on an unknown exe. No source, no debug info.
could be either but, in either case the program's goal is to generate very accurate information about the code being analyzed.

You disassemble, you analyse, and you take guesses.
Correct.  The basic idea is to validate the consequences of the guesses made by verifying consistency among them. 

And then you add that as dwarf, and let the user run it in a debugger, and the user can then inspect the data, and amend the guesses.
I am being very ambitious.  One of the reasons I decided to try a new approach is because I'd like the program to do _all_ the work.  Yes, that's quite likely unreasonable but, there is no doubt in my mind that the results obtained by IDA Pro (I use that as the benchmark) can be _greatly_ surpassed. 

My first "cut" will be fully static.  There won't even be a mechanism for the user to provide information/guesses to the analytical engine.  Once I have figured out the cases where it is truly impossible for the engine to generate the info then, I'll give a "door" for a user to give some input.  Of course, there are cases where it is unavoidable to have the user create decent variable names and provide those to the analytical engine but, even there, I want the analysis to be so thorough and complete that, at least in some cases, the engine will be able to generate genuinely informative variable names which, of course, will become part of the debugging symbols applicable to the program.

Now your plan is to write debug info in an existing format, and use an existing debugger.
I probably won't use an existing debugger because, doing so means I inherit all its limitations.  Once I've seen what the analytical engine can figure out then I can write a debugger tailored specifically to enable a human being to figure out what the analytical engine couldn't.

Within FpDebug you have access to controlling an exe in memory (lots of it work without debug info, some wants debug info ... but that does not need to come from a file. (And if/where FpDebug does not have proper separation, that could/should probably be fixed anyhow).
currently, I use FpDebug, VS Studio's debugger and IDA Pro's debugger to acquire knowledge in the various areas needed (such as acquiring a reasonably solid understanding of DWARF.)

One of the downsides of FpDebug (and GDB) which I have mentioned in the past is, neither is really convenient when debugging at the assembly level and, while I don't need that feature right now, as the project progresses, solid assembly code management will become very important (IDA Pro is quite good at that... I want to go beyond that.)

All the "debugging info" either in form of guesses or verified facts will be stored in memory.  The program will need a _lot_ of memory.

Thank you for all those thoughts... no doubt I'll have more questions for you as the project advances (it will be slow... probably as long as six months before I have a "proof of concept" I deem satisfactory.)

Stated succinctly, I want to develop a disassembler/debugger/analyzer that is noticeably superior, more capable and easier to use than any other offering at this time.  A tall order.   That said, I have no intention to make it a tool to reverse engineer malware.  It's purpose will be to be used on "normal" software (such as Windows system dlls and/or programs.)

« Last Edit: November 29, 2022, 10:59:48 pm by 440bx »
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Lazarus, FpDebug and GDB
« Reply #22 on: November 30, 2022, 01:43:14 am »
Quote
if I am understanding you correctly, it seems that FpDebug's ultimate goal is not just being a debugger for FPC but also one that could be used for any other GNU language.  Is my impression correct ?

See earlier FpDebug <> LazDebuggerFp

Yes for FpDebug (well 99% or so). Though active testing is with FPC.

FpDebug implements
- Dwarf reader - independent of compiler (mostly / but hopefully nothing that  would stop reading dwarf from other compilers)
  : Could be replaced by any other debug info provider (well, may need some fixes, but the design for this is in place)
- Optional unit (must be used explicitly) for FPC extended Dwarf stuff
- General control of exe (run / pause / step)  / LazDebuggerFp has extensions for FPC)
- MemReader
- Breakpoints
- Pascal parser for watch expressions / but again, this needs to be chosen by app / other parsers can be added
- Some helper code used by the IDE

Quote
One of the downsides of FpDebug (and GDB) which I have mentioned in the past is, neither is really convenient when debugging at the assembly level
Yes well, would be nice if added (e.g. link for any address found in asm...)

Btw, half of the deal is IDE, not FpDebug. All display and user interaction is controlled by the IDE.

 

TinyPortal © 2005-2018