Moving away from GDB

[rvk]

Probably the order is important with -gl triggering dwarf2, overriding an earlier -gw3

Note that combinations are also possible, it would be interesting to see what -gw3l  and -glw3 resp. do

Nope, not for me. If I put the -gl after the -gw3 I still get DARF 3.

fpc -odummy_ dummy.pas
fpc -gl -odummy_gl dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw2 -gl -odummy_gl_gw2 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw3 -gl -odummy_gl_gw3 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
fpc -gw2 -odummy_gw2 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw3 -odummy_gw3 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
116035507 226696 dummy_
2093630322 1076400 dummy_gw2
2856255440 1139768 dummy_gl
2856255440 1139768 dummy_gl_gw2
3552731905 1139768 dummy_gl_gw3
3883557931 1076400 dummy_gw3

And the combination -gw3 with -gl

fpc -glw3 -odummy_glw3 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
fpc -gw3l -odummy_gw3l dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
3552731905 1139768 dummy_glw3
3552731905 1139768 dummy_gw3l

PS. I did this on arm64. I will try on amd64 (but the download of my bootstrap is very slow at the moment. WiFi grrr).

[rvk]

Result on my newly installed Debian x86_64 (amd64).

This is indeed different from the arm64 result.

fpc -odummy_ dummy.pas
fpc -gl -odummy_gl dummy.pas
  Compilation Unit @ offset 0:
   Length:        0x2611 (32-bit)
   Version:       3
fpc -gw2 -gl -odummy_gl_gw2 dummy.pas
  Compilation Unit @ offset 0:
   Length:        0x2611 (32-bit)
   Version:       3
fpc -gw3 -gl -odummy_gl_gw3 dummy.pas
  Compilation Unit @ offset 0:
   Length:        0x2611 (32-bit)
   Version:       3
fpc -gw2 -odummy_gw2 dummy.pas
  Compilation Unit @ offset 0:
   Length:        0x2611 (32-bit)
   Version:       3
fpc -gw3 -odummy_gw3 dummy.pas
  Compilation Unit @ offset 0:
   Length:        0x2611 (32-bit)
   Version:       3
1036446534 1035608 dummy_gw2
152834824 1089568 dummy_gl_gw2
222374223 1035616 dummy_gw3
4248296208 1089568 dummy_gl
4248296208 1089568 dummy_gl_gw3
4276196424 206736 dummy_

My fpc itself, compiled with the same script, is now also DWARF 3 
(my script options are -g -gl -O2 -Xs -CX -XX -v0 and it seems to compile version 3 ??)

Code: Pascal  [Select][+][-]

1. $ readelf --debug-dump=info ~/dev/fpc/bin/fpc | head | grep -A 2 'Compilation Unit @'
2.   Compilation Unit @ offset 0:
3.    Length:        0x2611 (32-bit)
4.    Version:       3

I'll update my arm version to see if the result changes.

[rvk]

With a freshly compiled version of fpc trunk on arm64.
(so definitely something different with amd64)

This seems correct:

fpc -odummy_ dummy.pas
fpc -gl -odummy_gl dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw2 -gl -odummy_gl_gw2 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw3 -gl -odummy_gl_gw3 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
fpc -gw2 -odummy_gw2 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x77 (32-bit)
   Version:       2
fpc -gw3 -odummy_gw3 dummy.pas
  Compilation Unit @ offset 0x0:
   Length:        0x78 (32-bit)
   Version:       3
1102420958 1059136 dummy_gw2
1311066681 1118416 dummy_gl
1311066681 1118416 dummy_gl_gw2
244809924 1118416 dummy_gl_gw3
2735510098 1059136 dummy_gw3
3887319730 226696 dummy_

Code: Bash  [Select][+][-]

1. $ readelf --debug-dump=info ~/dev/fpc/bin/fpc | head | grep -A 2 'Compilation Unit @'
2.   Compilation Unit @ offset 0x0:
3.    Length:        0xfad (32-bit)
4.    Version:       2

[Martin_fr]

Just a note: you can compile each unit with a different dwarf version (you can probably even mix dwarf and stabs).

If your compiler and rtl is build with dwarf-2, and you compile your app with dwarf-3 then you still use the ppu and o files from your pre-build rtl (which have dwarf-2).

So, you should expect a mix of versions.

In principal such a mix is perfectly fine. The debugger can show the data for each unit.

Using FpDebug there is an upside when having "dwarf 3 only". => Dwarf does not make a distinction between a "AnsiString" and "PChar" (or in some cases "array of char"). To be exact, it does not know the Pascal string type.

Fpc makes tiny differences encoding those types. And FpDebug reacts to those internals. => If you have any Dwarf-2 in the mix, then FpDebug can't do that, and you may get results that have 2 values (FpDebug returns Pchar and string / if/when it makes a diff).

Note that (on linux), even if you have a string in a unit that has dwarf-3, if your rtl has dwarf-2 you still get the issues, because the unit in your project may have pointers into the dwarf data from the rtl, and therefore includes the dwarf-2 info.

Btw Fpc has -gw4  (at least in trunk)

[MarkMLl]

I'm off this for at least a few hours, but will be back. However

Probably the order is important with -gl triggering dwarf2, overriding an earlier -gw3

No, definitely not. I'm explicitly using -gl -gw3 as shown at the start of the script run, and I'm logging the actual parameters as part of my FPC build (I put the script I use at https://github.com/MarkMLl/fpclaz_build_scripts earlier today).

MarkMLl

[rvk]

I thought the first entry was always the main program.
It seems to be that on amd64 the first unit is NOT the main program.

What is that si_prc.pp ??
And it seems that on amd64 there is always some units with version 3 (while I only have -gl in my options).

Code: Bash  [Select][+][-]

1. rik@XPS17:~/d$ fpc -gw2 dummy
2. Free Pascal Compiler version 3.3.1 [2023/09/01] for x86_64
3. Copyright (c) 1993-2023 by Florian Klaempfl and others
4. Target OS: Linux for x86-64
5. Compiling dummy.pas
6. Linking dummy
7. 5 lines compiled, 0.1 sec, 145520 bytes code, 58912 bytes data
8. rik@XPS17:~/d$ readelf --debug-dump=info dummy | grep -A 12 'Compilation Unit @' | grep "Version\|DW_AT_name"
9.    Version:       3
10.     <c>   DW_AT_name        : si_prc.pp
11.    Version:       2
12.     <2621>   DW_AT_name        : dummy.pas
13.    Version:       3
14.     <2691>   DW_AT_name        : system.pp

On arm64 the first is the main program:

Code: Bash  [Select][+][-]

1. root@8d805006b2cd:~/d# fpc -gw2 dummy
2. Free Pascal Compiler version 3.3.1 [2023/09/01] for aarch64
3. Copyright (c) 1993-2023 by Florian Klaempfl and others
4. Target OS: Linux for AArch64
5. Compiling dummy.pas
6. Assembling test
7. Linking dummy
8. 4 lines compiled, 0.3 sec, 166724 bytes code, 61042 bytes data
9. root@8d805006b2cd:~/d# readelf --debug-dump=info dummy | grep -A 12 'Compilation Unit @' | grep "Version\|DW_AT_name"
10.    Version:       2
11.     <c>   DW_AT_name        : dummy.pas
12.    Version:       2
13.     <87>   DW_AT_name        : system.pp

[rvk]

New version with all units:
That seems to confirm the main program is compiled with DWARF 2.
But it is puzzling my whole FPC install is DWARF 3 (without giving that option).

But seeing the first with only -gl is version 3 I think the default changed to DWARF 3 on AMD64 (and not yet on ARM64).

Code: Bash  [Select][+][-]

1. fpc -odummy_ dummy.pas
2. fpc -gl -odummy_gl dummy.pas
3.    Version:       3
4.     <c>   DW_AT_name        : si_prc.pp
5.    Version:       3
6.     <2621>   DW_AT_name        : dummy.pas
7.    Version:       3
8.     <2692>   DW_AT_name        : system.pp
9.    Version:       3
10.     <47d48>   DW_AT_name        : ../inc/lnfodwrf.pp
11.    Version:       3
12.     <49d4b>   DW_AT_name        : ../inc/exeinfo.pp
13.    Version:       3
14.     <4ac0a>   DW_AT_name        : ../inc/strings.pp
15. fpc -gw2 -gl -odummy_gl_gw2 dummy.pas
16.    Version:       3
17.     <c>   DW_AT_name        : si_prc.pp
18.    Version:       2
19.     <2621>   DW_AT_name        : dummy.pas
20.    Version:       3
21.     <2691>   DW_AT_name        : system.pp
22.    Version:       3
23.     <47d47>   DW_AT_name        : ../inc/lnfodwrf.pp
24.    Version:       3
25.     <49d4a>   DW_AT_name        : ../inc/exeinfo.pp
26.    Version:       3
27.     <4ac09>   DW_AT_name        : ../inc/strings.pp
28. fpc -gw3 -gl -odummy_gl_gw3 dummy.pas
29.    Version:       3
30.     <c>   DW_AT_name        : si_prc.pp
31.    Version:       3
32.     <2621>   DW_AT_name        : dummy.pas
33.    Version:       3
34.     <2692>   DW_AT_name        : system.pp
35.    Version:       3
36.     <47d48>   DW_AT_name        : ../inc/lnfodwrf.pp
37.    Version:       3
38.     <49d4b>   DW_AT_name        : ../inc/exeinfo.pp
39.    Version:       3
40.     <4ac0a>   DW_AT_name        : ../inc/strings.pp
41. fpc -gw2 -odummy_gw2 dummy.pas
42.    Version:       3
43.     <c>   DW_AT_name        : si_prc.pp
44.    Version:       2
45.     <2621>   DW_AT_name        : dummy.pas
46.    Version:       3
47.     <2691>   DW_AT_name        : system.pp
48. fpc -gw3 -odummy_gw3 dummy.pas
49.    Version:       3
50.     <c>   DW_AT_name        : si_prc.pp
51.    Version:       3
52.     <2621>   DW_AT_name        : dummy.pas
53.    Version:       3
54.     <2692>   DW_AT_name        : system.pp
55. 1036446534 1035608 dummy_gw2
56. 152834824 1089568 dummy_gl_gw2
57. 222374223 1035616 dummy_gw3
58. 4248296208 1089568 dummy_gl
59. 4248296208 1089568 dummy_gl_gw3
60. 4276196424 206736 dummy_

[rvk]

And finally a more readable version: (I won't bother you with further spam )

On ARM64 (compiled with an FPC with options -gl)

Code: Bash  [Select][+][-]

1. root@8d805006b2cd:~/d# sh t
2. fpc  -odummy_ dummy.pas
3. --
4. fpc -gl -odummy_gl dummy.pas
5.    Version:       2 Unit         : dummy.pas
6.    Version:       2 Unit         : system.pp
7.    Version:       2 Unit         : ../inc/lnfodwrf.pp
8.    Version:       2 Unit         : ../inc/exeinfo.pp
9.    Version:       2 Unit         : ../inc/strings.pp
10. --
11. fpc -gw2 -gl -odummy_gw2_gl dummy.pas
12.    Version:       2 Unit         : dummy.pas
13.    Version:       2 Unit         : system.pp
14.    Version:       2 Unit         : ../inc/lnfodwrf.pp
15.    Version:       2 Unit         : ../inc/exeinfo.pp
16.    Version:       2 Unit         : ../inc/strings.pp
17. --
18. fpc -gw3 -gl -odummy_gw3_gl dummy.pas
19.    Version:       3 Unit         : dummy.pas
20.    Version:       2 Unit         : system.pp
21.    Version:       2 Unit         : ../inc/lnfodwrf.pp
22.    Version:       2 Unit         : ../inc/exeinfo.pp
23.    Version:       2 Unit         : ../inc/strings.pp
24. --
25. fpc -gw2 -odummy_gw2 dummy.pas
26.    Version:       2 Unit         : dummy.pas
27.    Version:       2 Unit         : system.pp
28. --
29. fpc -gw3 -odummy_gw3 dummy.pas
30.    Version:       3 Unit         : dummy.pas
31.    Version:       2 Unit         : system.pp
32. --
33. 1298906025 226696 dummy_
34. 162089040 1059192 dummy_gw2
35. 1739967246 1118464 dummy_gw3_gl
36. 2552954366 1059192 dummy_gw3
37. 3799279261 1118456 dummy_gl
38. 3799279261 1118456 dummy_gw2_gl

On AMD64 (compiled with an FPC with options -gl)

Code: Bash  [Select][+][-]

1. rik@XPS17:~/d$ sh t
2. fpc  -odummy_ dummy.pas
3. --
4. fpc -gl -odummy_gl dummy.pas
5.    Version:       3 Unit         : si_prc.pp
6.    Version:       3 Unit         : dummy.pas
7.    Version:       3 Unit         : system.pp
8.    Version:       3 Unit         : ../inc/lnfodwrf.pp
9.    Version:       3 Unit         : ../inc/exeinfo.pp
10.    Version:       3 Unit         : ../inc/strings.pp
11. --
12. fpc -gw2 -gl -odummy_gw2_gl dummy.pas
13.    Version:       3 Unit         : si_prc.pp
14.    Version:       2 Unit         : dummy.pas
15.    Version:       3 Unit         : system.pp
16.    Version:       3 Unit         : ../inc/lnfodwrf.pp
17.    Version:       3 Unit         : ../inc/exeinfo.pp
18.    Version:       3 Unit         : ../inc/strings.pp
19. --
20. fpc -gw3 -gl -odummy_gw3_gl dummy.pas
21.    Version:       3 Unit         : si_prc.pp
22.    Version:       3 Unit         : dummy.pas
23.    Version:       3 Unit         : system.pp
24.    Version:       3 Unit         : ../inc/lnfodwrf.pp
25.    Version:       3 Unit         : ../inc/exeinfo.pp
26.    Version:       3 Unit         : ../inc/strings.pp
27. --
28. fpc -gw2 -odummy_gw2 dummy.pas
29.    Version:       3 Unit         : si_prc.pp
30.    Version:       2 Unit         : dummy.pas
31.    Version:       3 Unit         : system.pp
32. --
33. fpc -gw3 -odummy_gw3 dummy.pas
34.    Version:       3 Unit         : si_prc.pp
35.    Version:       3 Unit         : dummy.pas
36.    Version:       3 Unit         : system.pp
37. --
38. 1036446534 1035608 dummy_gw2
39. 152834824 1089568 dummy_gw2_gl
40. 222374223 1035616 dummy_gw3
41. 4248296208 1089568 dummy_gl
42. 4248296208 1089568 dummy_gw3_gl
43. 4276196424 206736 dummy_

Adjusted script:

Code: Bash  [Select][+][-]

1. #!/bin/sh
2.  
3. compile_and_check() {
4.   echo "fpc $2 -o$1 dummy.pas"
5.   fpc $2 -o$1 dummy.pas > /dev/null
6.   readelf --debug-dump=info $1 | grep -A 12 'Compilation Unit @' | grep "Version\|DW_AT_name" | perl -0777 -pe 's/\n.*DW_AT_name/ Unit $1/g'
7.   echo "--"
8. }
9.  
10. compile_and_check dummy_ ""
11. compile_and_check dummy_gl "-gl"
12. compile_and_check dummy_gw2_gl "-gw2 -gl"
13. compile_and_check dummy_gw3_gl "-gw3 -gl"
14. compile_and_check dummy_gw2 "-gw2"
15. compile_and_check dummy_gw3 "-gw3"
16.  
17. cksum dummy_* | sort
18. rm dummy_*

AMD64 definitely defaults to DWARF 3 while ARM64 does not.

[TRon]

I thought the first entry was always the main program.

It seems to be that on amd64 the first unit is NOT the main program.

As you discovered yourself, no (usually not for any platform/target).

What is that si_prc.pp ??

Startup & initialization code.

Before your main program is entered, the startup code also takes into account the unit initializations/finalizations but more importantly the startup code itself. Every platform/target has its own way of pushing important information before your program startup into registers and/or the stack. This differs for each platform and is even dependable whether or not libc is used (that uses its own startup code that has to be taken into account). Also before your program is able to run the system unit needs to initialized so that it is able to work on/for that target (which depends on target specific information).

That is why there also exist variations of the startup code.

[MarkMLl]

In theory that could be extended to gdbserver. But no plans for that at the moment.
Also: It's considerable slower.

I've found gdbserver indispensable when debugging anything which uses low-numbered TCP ports. During normal operation access can be granted using POSIX capabilities, but since these are lost if started using a debugger the only realistic way is to run as root hence:

Code: Text  [Select][+][-]

1. # gdbserver :2345 ./Watchxxx-x86_64-linux-gtk2
2.  

At that point the IDE can attach to port 2345, and the program being tested barely knows it's there.

I know it's a long shot, but would it be possible to build a variant of FpDebug which behaved like gdbserver, probably with an opaque signalling protocol?

MarkMLl

[MarkMLl]

If I build FPC (i.e. including runtimes etc.) for DWARF 4 and then run the test script I posted earlier, I get

Code: Text  [Select][+][-]

1. FPC compiler/runtimes built with -gl -gw4.
2. fpc -odummy_ dummy.pas
3. fpc -gl -odummy_gl dummy.pas
4.   Compilation Unit @ offset 0:
5.    Length:        0x1eda (32-bit)
6.    Version:       4
7. fpc -gl -gw2 -odummy_gl_gw2 dummy.pas
8.   Compilation Unit @ offset 0:
9.    Length:        0x1eda (32-bit)
10.    Version:       4
11. fpc -gl -gw3 -odummy_gl_gw3 dummy.pas
12.   Compilation Unit @ offset 0:
13.    Length:        0x1eda (32-bit)
14.    Version:       4
15. fpc -gw2 -odummy_gw2 dummy.pas
16.   Compilation Unit @ offset 0:
17.    Length:        0x1eda (32-bit)
18.    Version:       4
19. fpc -gw3 -odummy_gw3 dummy.pas
20.   Compilation Unit @ offset 0:
21.    Length:        0x1eda (32-bit)
22.    Version:       4
23. 3427178416 1029376 dummy_gl_gw3
24. 3447956569 231872 dummy_
25. 3581180241 975368 dummy_gw2
26. 3719963618 1029376 dummy_gl
27. 3719963618 1029376 dummy_gl_gw2
28. 656825139 975368 dummy_gw3
29.  

So if nothing else it appears to imply that using readelf is a fairly reliable way of getting the basic DWARF version even if the interplay between the various compile-time options might possibly be slightly non-intuitive (I'm certainly losing track of quite where we're at :-)

Having said which,

Code: Text  [Select][+][-]

1. $ readelf --debug-dump=info dummy_gw2 | grep -A 2 'Compilation Unit @'
2.   Compilation Unit @ offset 0:
3.    Length:        0x1eda (32-bit)
4.    Version:       4
5. --
6.   Compilation Unit @ offset 0x1ede:
7.    Length:        0x6f (32-bit)
8.    Version:       2
9. --
10.   Compilation Unit @ offset 0x1f51:
11.    Length:        0x3c006 (32-bit)
12.    Version:       4
13.  

So the versioning exercise really boils down to looking at both the RTL's startup code which could be reasonably expected to determine the compiler's choice of default DWARF version, and the version which has actually been used for the application code.

For this particular system based on Debian "Bookworm" x86_64, I'm going to build FPC with DWARF 3 then ditto the Lazarus IDE. Once I've done that, I'll see how happy the IDE is with gdb and FpDebug.

Noting Martin's comment that in principle the debugger should be happy with a mix of DWARF (and possibly even stabs) versions, in practice IDE+gdb appears unhappy with DWARF 2 and I think somebody's said that FpDebug wanted DWARF 3 which seems to contradict his position. And while I'm not a heavy user of Lazarus packages (i.e. via FpUpDeluxe etc.) the possibility of a package carrying an incompatible DWARF-version option around with it is slightly troubling.

Plus there's still also that dialogue which told me to specify a DWARF version (including DWARF 2) without giving any information as to why what a binary was actually using (which happened to be DWARF 2) was considered to be a problem.

MarkMLl

[rvk]

If I build FPC (i.e. including runtimes etc.) for DWARF 4 and then run the test script I posted earlier,

The command I initially used only shows the first unit in the executable.
But on AMD64, the first unit is a bootstrap si_prc.pp, which could have a different DWARF version.

So you need to use the adjusted script from a few posts back to get a clear picture of all used versions per unit.

So if for the stability you need a specific version of DWARF for all units you would need to recompile all those units.

[MarkMLl]

As you say, there were quite a few :-)

I've just got to the same place using

Code: Text  [Select][+][-]

1. $ readelf --debug-dump=info dummy_gl_gw2 | grep -A 7 'Compilation Unit @' | egrep 'Version|_AT_'
2.  

which also shows the compiler version: I think picking up packages built elsewhere could be useful.

Using Perl for this is an obvious no-brainer, although the majority of Pascal users seem to view Perl with the disdain that the majority of Perl users reserve for Pascal :-)

MarkMLl

[rvk]

Using Perl for this is an obvious no-brainer, although the majority of Pascal users seem to view Perl with the disdain that the majority of Perl users reserve for Pascal :-)

Yeah, I wanted to show both DWARF version and unit on one line so it would be a much nicer output. I couldn't get that done with just the grep command 

What problems are there with fpdebug and mixing DWARF 2 and 3?

[MarkMLl]

What problems are there with fpdebug and mixing DWARF 2 and 3?

First problem was that as soon as I tried to switch to fpdebug it presented me with a dialogue telling me to specify a DWARF version. One of the options was DWARF 2, which I thought (and still think) I was using. At that point I started looking at determining what was actually in the binary...

However I've just discovered problems trying to do a complete Lazarus 2.2.4 build using my DWARF3-enabled FPC. I'll be back once I know what I'm looking at...

MarkMLl