Recent

Author Topic: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support  (Read 4038 times)

Graeme

  • Hero Member
  • *****
  • Posts: 1518
    • Graeme on the web
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #15 on: April 07, 2026, 01:01:42 pm »
I have a question at this time.  Does the _current_ OPDF support block scoped inline variables

Good question, and an interesting one to chase — but with a small twist: As far
as I know FPC 3.3.1 doesn't actually support Delphi-style inline var x: Integer
:= 5; declarations yet, in any mode. I just tested it with {$mode delphi} to be
sure, and the parser rejects it outright. So the question is currently
hypothetical for FPC, regardless of debug format. Or did incorrectly interpret
what you meant?
                                                                                                                                 
That said, adding proper block-scope support when the time comes would be a
small format extension rather than a redesign: a new recBlockScope record
carrying a parent ScopeID and a LowPC/HighPC range, and the FPC writer teaching
the local-var emitter to point at the right scope.

Regards,
  Graeme
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

440bx

  • Hero Member
  • *****
  • Posts: 6382
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #16 on: April 07, 2026, 01:27:48 pm »
<snip>
I just tested it with {$mode delphi} to be sure, and the parser rejects it outright. So the question is currently hypothetical for FPC, regardless of sebug format.
<snip>
                                                                                                                                 
That said, adding proper block-scope support when the time comes would be a small format extension rather than a redesign: a new recBlockScope record carrying a parent ScopeID and a LowPC/HighPC range, and the FPC writer teaching the local-var emitter to point at the right scope.
What brought that question to my mind is the FPC branch from @Fibonacci.   You are correct that the current branch does not support inline variables but his does (very recent addition.)  Obviously I cannot speak for him but, he might be interested in adding the improvements in your branch to his.

On a different note, I read your reply to @Martin where you mentioned that, at least at this time, OPDF isn't a replacement for DWARF.  I just wanted to say that, IMO, it would be great if it _eventually_ could be a full replacement because, by the very fact that it is tailored to Pascal, it has much better long term potential for FPC debugging than DWARF does.  I fully understand that using DWARF was originally a sound decision since it provided a debugger (GDB) usable with FPC but, now that there is FpDebug, I believe that planning to eventually part ways with GDB/DWARF is good planning and will be a plus in the future (presuming, of course, that OPDF grows with FPC, which I believe is the raison d'etre of OPDF.)

Strictly for personal curiosity and because I am Windows centric, can you provide some rough estimate when the first version of OPD may be available for Windows ?
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12303
  • Debugger - SynEdit - and more
    • wiki
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #17 on: April 07, 2026, 02:24:14 pm »
The thing your question changes for me, and the reason it's well-timed: OPDF is
still in pull-request state with the FPC team. The spec is genuinely still
malleable, with no deployed binaries to stay compatible with. So if extending
TDefAnsiString to carry header field offsets — LengthOffset, RefCountOffset,
CodePageOffset, ElementSizeOffset — is the right call, I can put them straight
into v1 of the record before the PR lands. The FPC writer (best placed to know
the exact offsets) emits them at compile time, and any consumer reads them at
runtime with no recompile and no per-FPC constant table.
In that case there is a decision for you to make. (well I don't know how you encode the type).

Assuming that you just tag each variable (each data) with "Type_AnsiString", then
- add that info for each occurrence
- or add it once for the type (global)...

Both isn't ideal. The 3rd is:

Allow to define 1 or more types of Type_AnsiString
And then tag the data wich those "dynamic" types.

Disadvantage:
- One step of indirection

Advantage
- If FPC ever has diff settings in one exe..... (e.g. when you declare types for codepages)
 That currently is unlikely. And one exe can't mix diff FPC versions.
  Though technically I don't know if stuff could be linked together...


Also, even if you go for one global: your engine needs more than one global. Because a dynamic library can definitely have diff values. A library can be build with a diff FPC version.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12303
  • Debugger - SynEdit - and more
    • wiki
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #18 on: April 07, 2026, 02:28:48 pm »
I fully understand that using DWARF was originally a sound decision since it provided a debugger (GDB) usable with FPC but, now that there is FpDebug, I believe that planning to eventually part ways with GDB/DWARF is good planning and will be a plus in the future (presuming, of course, that OPDF grows with FPC, which I believe is the raison d'etre of OPDF.)

That would require to have replacements for all CPU architectures that FPC supports.

It also would prevent people from debugging into libraries (such as gtk or kernel). And that may well be needed.
Well, unless you implement a debugger (for each arch) that can read OPDF and DWARF

440bx

  • Hero Member
  • *****
  • Posts: 6382
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #19 on: April 07, 2026, 08:58:15 pm »
That would require to have replacements for all CPU architectures that FPC supports.

It also would prevent people from debugging into libraries (such as gtk or kernel). And that may well be needed.
Well, unless you implement a debugger (for each arch) that can read OPDF and DWARF
Strictly theoretically speaking and, emphasizing that I said "eventually", it isn't difficult to have debug symbols cater to all CPU architectures, actually, the definition of debug symbols should be _largely_, (very unlikely completely) independent of CPU architectures.  DWARF is an example of that.

But, you are correct that it would require the implementation of debuggers capable of debugging privileged code in addition to user mode code.  For the record, I'm not suggesting that FpDebug in any way attempt to fulfill that function, just that the debug symbol definitions should not be a limiting factor.
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12303
  • Debugger - SynEdit - and more
    • wiki
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #20 on: April 07, 2026, 09:24:21 pm »
Well, a bit offtopic here, DWARF is actually no more limiting than OPDF.
Except FPC does not fully use it.

1) DWARF has dedicated space for custom additions. So if OPDF is a complete custom format, all it can do can be done in DWARF too. (if someone implements it, and the FPC team accepts it).

2) Yes DWARF may (trusting Graeme's word on OPDF) need a bit more work. For some more complex values it describes the location as an "expression" a sort of "math script" that needs to be interpreted.
But, we already have (much of) that part.

3) DWARF is open. I submitted definitions for Pascal property. DWARF now has them (will have with the next release). Implementations are lacking.



(Trusting Graeme) OPDF may make it easier to implement. Though the limiting factor for me, when looking at e.g. adding stuff to FPC is not the DWARF site, but the compiler internals. Those also take time to get familiar with...

In any case having several debuggers to chose from is good. So having OPDF is great. But only if it lives in parallel. Otherwise we are back to were we started.

I am still trying to keep even the gdb backend going. Though I have very little time for that, and do little active testing at current.

I am also trying to make time, to clean the IDE to have a proper interface for debugger backends. => so then when that is done, anyone could add an OPDF backend. (and it would have the full frontend the IDE already offers). Even create backend combining several formats ...

But it all lives on having choice.

(and just then, there also is valgrind, wouldn't wont to miss that)




While not having tested it, I would expect OPDF to have a smaller footprint, so for anyone needing that it will be great. (I haven't looked at some of the new space savers in later DWARF versions, but they will be a lot of work in comparison / if the even get anywhere)


Graeme

  • Hero Member
  • *****
  • Posts: 1518
    • Graeme on the web
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #21 on: April 08, 2026, 10:32:59 pm »
                                                                                                                                 
What brought that question to my mind is the FPC branch from @Fibonacci.
Ah, that makes sense. And so the FPC fracmentation begins. :-/ I warned the FPC team that could happen.

Quote
On a different note, I read your reply to @Martin where you mentioned that, at least at this time, OPDF isn't a replacement for DWARF.  I just wanted to say that, IMO, it would be great if it _eventually_ could be a full replacement

There are a lot of different use cases to consider, because FPC supports a LOT of platforms and CPU architectures. So DWARF will still be there for the long term. You should be able to even have both DWARF and OPDF headers in a single binary, and they should not interfere with each other.

Quote
Strictly for personal curiosity and because I am Windows centric, can you provide some rough estimate when the first version of OPD may be available for Windows ?

It's next on my To Do list, after I finished the OPDF debugger integration in fpGUI's lightweight IDE. Shouldn't be too long, before I start working on Windows support - and it should be a relatively thin layer of code to add. [last famous words :-D]
« Last Edit: April 09, 2026, 12:59:30 am by Graeme »
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

Graeme

  • Hero Member
  • *****
  • Posts: 1518
    • Graeme on the web
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #22 on: April 09, 2026, 01:24:14 am »
In that case there is a decision for you to make. (well I don't know how you encode the type).
Hi Martin,

That's a really thoughtful breakdown of the options—the dynamic library angle, in particular, is a key insight worth addressing directly.

To show how the spec currently handles this, here are two relevant type records (other string types are also handled):

ShortString (entirely inline, no heap pointer, no header fields needed):

Code: Pascal  [Select][+][-]
  1. TDefShortString = packed record
  2.   TypeID: TTypeID;    // 4 bytes
  3.   MaxLength: Byte;    // 1 byte — maximum capacity (0..255)
  4.   NameLen: TNameLen;  // 2 bytes
  5.   // Followed by: Name (NameLen bytes)
  6. end;
  7.  

AnsiString (carries all four header offsets, emitted by the compiler at build time):

Code: Pascal  [Select][+][-]
  1. TDefAnsiString = packed record
  2.   TypeID: TTypeID;             // 4 bytes
  3.   LengthOffset: SmallInt;      // 2 bytes - signed offset from data ptr
  4.   RefCountOffset: SmallInt;    // 2 bytes
  5.   CodePageOffset: SmallInt;    // 2 bytes
  6.   ElementSizeOffset: SmallInt; // 2 bytes
  7.   NameLen: TNameLen;           // 2 bytes
  8.   // Followed by: Name (NameLen bytes)
  9. end;
  10.  

As you can see, the "compiler emits offsets at build time, consumer reads them at runtime" design is already baked into the spec. There are no hardcoded platform constants [as of two days ago ;-)].

Regarding the dynamic library concern: Each ELF binary (executable or .so) carries its own .opdf section with its own type pool. A shared library built against a different FPC version will emit its own TDefAnsiString record with its own offsets, completely independent of the host executable. The debugger/consumer reads each section separately, so "multiple globals" are naturally handled at the binary boundary.

Within a single binary, all AnsiStrings share a single memory layout regardless of codepage, so one record per binary remains correct for now.

Your "Option 3" (multiple AnsiString subtypes) is definitely worth keeping in mind for a hypothetical future where layouts might diverge within a single unit, but I don't think the indirection cost is justified just yet. The format is extensible enough that we can add that layer later if the FPC implementation changes.
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

creaothceann

  • Sr. Member
  • ****
  • Posts: 335
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #23 on: April 09, 2026, 09:36:49 am »
ShortString (entirely inline, no heap pointer, no header fields needed):

Code: Pascal  [Select][+][-]
  1. TDefShortString = packed record
  2.   TypeID: TTypeID;    // 4 bytes
  3.   MaxLength: Byte;    // 1 byte — maximum capacity (0..255)
  4.   NameLen: TNameLen;  // 2 bytes
  5.   // Followed by: Name (NameLen bytes)
  6. end;
  7.  

A minor point, but I'd switch around MaxLength and NameLen, to position NameLen at its natural alignment. (Or add a reserved byte before/after MaxLength for future use and/or padding.)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12303
  • Debugger - SynEdit - and more
    • wiki
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #24 on: April 09, 2026, 09:52:28 am »
Within a single binary, all AnsiStrings share a single memory layout regardless of codepage, so one record per binary remains correct for now.

Makes sense for the header layout...

Is "NameLen" + Bytes the type name? (or the var name?)

I guess the type name, since the record does not have an address (as a var would need).
What if a user has multiply types? I know they can all may into one single type, and if they are just aliases fpc does that. Otherwise, well it rarely matters.



You may want to test untyped string constants. They should work, but just make sure they are handled.
Because while the compiler has to put them into the exe, the compiler will not allow to take the address of them (or any char within).



And a new question.

How do you handle modifications of strings through the debugger? (as, depending on type you need to trigger copy on write, if the ref count demands).

Hardcoded behaviour in the debugger? (just find free/allocmem in the exe)?
Or find fpc_ansi_assign and variants?

And with more managed types coming up in 3.3.1 more of that question arises.

It might be an idea to consider including pointers/descriptions of the relevant setter methods in the debug info (together with the type).
Though, that might require info on how to call them (which param goes where, in which order...)

Also maybe marking if a string var is "const" param => as changing that would have more side effects than the user may expect.

Graeme

  • Hero Member
  • *****
  • Posts: 1518
    • Graeme on the web
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #25 on: April 09, 2026, 06:49:39 pm »
Makes sense for the header layout...

Maybe it's easier to read the OPDF spec.  It's great bedtime reading material. ;)
https://github.com/graemeg/opdebugger/blob/master/docs/opdf-specification.adoc#3-binary-format


And then lets use a simple example program:

Code: Pascal  [Select][+][-]
  1. program UntypedStringDemo;
  2.  
  3. const
  4.   // This is an untyped string constant
  5.   GREETING = 'Hello, Free Pascal World!';
  6.   VERSION  = 'v3.3.1';
  7.  
  8. var
  9.   UserGreet: string;
  10.  
  11. begin
  12.   // Using the untyped constant to initialize a variable
  13.   UserGreet := GREETING;
  14.  
  15.   WriteLn(UserGreet);
  16.   WriteLn('Running Version: ', VERSION);
  17.  
  18.   // You can also access individual characters like an array
  19.   WriteLn('First letter of GREETING: ', GREETING[1]);
  20.  
  21.   // High() gives the declared length of the string constant
  22.   WriteLn('Length of the constant: ', Length(GREETING));
  23. end.
  24.  

Compile that with `fpc -gO UntypedStringDemo.pas` and then lets examine the debug info using the opdf_dump tool.

Code: [Select]
$ opdf_dump -t Constant ./UntypedStringDemo
OPDF Section Analysis: UntypedStringDemo
Section size: 23.0 KB

  [356] Constant (size=42):     TypeID=-268435243 Kind=1 ValueLen=25 Name="GREETING" Value=[48 65 6C 6C 6F 2C 20 46 72 65 65 20 50 61 73 63...]
  [357] Constant (size=22):     TypeID=-268435241 Kind=1 ValueLen=6 Name="VERSION" Value=[76 33 2E 33 2E 31]

=== Summary ===
Units:              1
Total records:      369

  Primitive:             46
  GlobalVar:              1
  ShortString:            3
  AnsiString:             3
  UnicodeString:          2
  Pointer:              127
  Array:                 83
  Record:                47
  Class:                 14
  Property:               6
  LineInfo:               7
  FunctionScope:          3
  Interface:              5
  Enum:                  13
  Set:                    6
  UnitDirectory:          1
  Constant:               2

Unique TypeIDs:     349  (349 type records — 1.0x duplication)
Section size:       23.0 KB
Header overhead:    32 bytes (1 headers x 32 bytes)
Est. reader memory: 68.8 KB (if all loaded into cache)
« Last Edit: April 09, 2026, 06:53:58 pm by Graeme »
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

440bx

  • Hero Member
  • *****
  • Posts: 6382
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #26 on: April 10, 2026, 03:58:18 am »
I've been reading the documentation and, I have a few comments:

the documentation says:
Quote
2.1. Decision: PasBuild Multi-Module Layout

+-- docs/                            # Documentation
¦   +-- opdf-specification.adoc
¦   +-- analysis.adoc
¦   +-- implementation-plan.adoc
¦   +-- design-decisions.adoc
¦   +-- progress.adoc
but, the above tree does NOT match the current docs directory.

where it says:
Quote
TypeID Word (65K) Cardinal (4B) Large projects with generics may need 100K+ type IDs
The "Cardinal (4B)" should be "Cardinal (4GB)".  This would it make consistent with previous lines in the same group of changes.


consider changing:
Quote
set Count = 10
to
Quote
set Count := 10
for consistency with the Pascal language and eliminate the apparent boolean ambiguity.  Strictly speaking assignment and equality are not the same thing (I'm sure you know that.)

question: how are array items set ? individually e.g, SomeArray[3] = SomeValue forcing individual assigments for each array element or is something like this, SomeArray[1..3] = (1, 2, 3) available ?  (note: hopefully the "=" will be changed to ":=" as suggested above.)  Also, is there a simple (one command) way to set a range of items to the same value, something like, SomeArray[1..3] = (all 2) ? to set all values to 2. It would be nice to have a "variant" of this command set all array items to all the same value in one shot (IOW, a way to initialize the entire array to a specified value... something which is occasionally very useful during debugging.)

In the following example given in the documentation:
Quote
Expression-condition breakpoints (e.g., break foo.pas:N if I > 100 and J < 5)
aren't parentheses required to separate the boolean sub-expressions, i.e, (I > 100) and (J < 5) ?

The whole thing seems to be very well thought out.  I'm looking forward to playing with the Windows version.
« Last Edit: April 10, 2026, 04:00:41 am by 440bx »
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

PascalDragon

  • Hero Member
  • *****
  • Posts: 6381
  • Compiler Developer
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #27 on: April 10, 2026, 07:37:59 am »
Regarding the dynamic library concern: Each ELF binary (executable or .so) carries its own .opdf section with its own type pool. A shared library built against a different FPC version will emit its own TDefAnsiString record with its own offsets, completely independent of the host executable. The debugger/consumer reads each section separately, so "multiple globals" are naturally handled at the binary boundary.

Do you have an idea how it will work with dynamic packages where types will be spread across modules? Especially if runtime packages are used the compiler can't add them to the pool generated at creation time of the binary.

Graeme

  • Hero Member
  • *****
  • Posts: 1518
    • Graeme on the web
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #28 on: April 11, 2026, 12:07:18 am »
First off, thank you for the feedback.

the documentation says:
Quote
2.1. Decision: PasBuild Multi-Module Layout

+-- docs/                            # Documentation
¦   +-- opdf-specification.adoc
¦   +-- analysis.adoc
¦   +-- implementation-plan.adoc
¦   +-- design-decisions.adoc
¦   +-- progress.adoc
but, the above tree does NOT match the current docs directory.

That decision document was/is a living document. A lot of it was written before I even wrote the first line of code. Then as I implemented things, some design choices changed, as I learned more or thought of future proofing. So I documented those ideas/changes as I went.

The only missing document from that list is the implementation-plan.adoc. And that was more of a personal To Do List, ticking things off as I go. Once I had my initial MVP in place, that document wasn't needed any more and got deleted.


Quote
where it says:
Quote
TypeID Word (65K) Cardinal (4B) Large projects with generics may need 100K+ type IDs
The "Cardinal (4B)" should be "Cardinal (4GB)".  This would it make consistent with previous lines in the same group of changes.

No, it's correct, but I can see the confusion, so I just updated it.

RecSize: Word is 16-bit unsigned, max value 65,535 — so max
record size of 64KB. Cardinal is 32-bit unsigned, max ~4.29 billion —
so max record size of 4GB. Both annotations are accurate.

TypeID: The values are mathematically correct — Word gives 65,535
possible IDs (~65K), and Cardinal gives ~4.29 billion. However, the
annotation Cardinal (4B) is ambiguous. In a size/storage context, "B"
typically means Bytes, so 4B reads as "4 Bytes" — which is actually
the storage size of a Cardinal, not its range. The intent was
"4 Billion IDs", but a reader could easily misread its meaning.

I've just updated it to ~4.3 billion.


Quote
consider changing:
Quote
set Count = 10
to
Quote
set Count := 10
for consistency with the Pascal language and eliminate the apparent boolean ambiguity.  Strictly speaking assignment and equality are not the same thing

It is a fair point that in Pascal = and :=
carry distinct meanings, and I can see why set Count = 10 might read
as an equality test at first glance.

That said, I think the set prefix already shifts the developer into
"debugger command" rather than "Pascal expression" territory, and =
follows the convention used by most debuggers (GDB, LLDB, PDB).
Switching to := would be a half-step towards Pascal syntax without
quite getting there — you would still be missing the trailing
semicolon, and set itself has no equivalent in the language.

Precedent from other debuggers:                                                                                                                 
  - GDB: set variable Count = 10     
  - LLDB: expr Count = 10                                                                                                                         
  - PDB: Count = 10

Quote
question: how are array items set ?
set only handles tcPrimitive and tcEnum types at the moment. Anything
else falls through to the else branch with "type not supported for assignment".

It is planned though, I just haven't gotten there yet. I've made a not of your other suggestions for set too.

Quote
Quote
Expression-condition breakpoints (e.g., break foo.pas:N if I > 100 and J < 5)
aren't parentheses required to separate the boolean sub-expressions, i.e, (I > 100) and (J < 5) ?
Pascal operator precedence already resolves this unambiguously. In Pascal, relational
operators (>, <, =, etc.) bind more tightly than and/or, so I > 100 and J < 5 is parsed
as (I > 100) and (J < 5) without parentheses.


Quote
The whole thing seems to be very well thought out.  I'm looking forward to playing with the Windows version.
Thank you - a lot of effort has gone into the design process. Hopefully you don't have to wait too much longer. :-)
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

440bx

  • Hero Member
  • *****
  • Posts: 6382
Re: [ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support
« Reply #29 on: April 11, 2026, 01:00:04 am »
Thank you Graeme, I appreciate the clarifications.
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

 

TinyPortal © 2005-2018