[ANN] Unofficial FPC 3.3.1 binary release with OPDF debugger support

[Graeme]

Hi all,

I'd like to share an unofficial pre-built binary release of FPC 3.3.1 for
Linux x86_64, with support for the Object Pascal Debug Format (OPDF) — a
new debug information format designed specifically for Free Pascal and
Object Pascal language features.

The release is available here:
https://github.com/graemeg/fpc-opdf-releases/releases


WHY OPDF?

DWARF is a general-purpose format and does not natively understand Object
Pascal concepts. OPDF addresses this by providing first-class support for:

  - Properties (field-backed and method-backed, including getter evaluation)
  - String types: ShortString, AnsiString, UnicodeString
  - Dynamic arrays with correct length display
  - Class instances with inherited fields and interfaces
  - Sets with enum member name display

OPDF data is embedded as an ELF .opdf section alongside your existing code.
It coexists with DWARF — you can use -gO and -gw together if you also need
GDB compatibility.


TRYING IT OUT

Download the archive from the releases page, extract it, and run the
bundled install.sh:

  tar -xzf fpc-3.3.1-unofficial-opdf-<date>-g<sha>-x86_64-linux.tar.gz
  cd fpc-3.3.1-unofficial-opdf-<date>-g<sha>-x86_64-linux/
  ./install.sh "$HOME/fpc-3.3.1-opdf"
  export PATH="$HOME/fpc-3.3.1-opdf/bin:$PATH"

Then compile your project with:

  fpc -gO myproject.pas

The reference command-line debugger (PDR) for OPDF is available at:
https://github.com/graemeg/opdebugger


CURRENT STATUS

PDR is a working CLI debugger (reference implementation) that reads OPDF
data and controls the debuggee via Linux ptrace. It supports breakpoints,
source-level stepping, call stack display, variable evaluation for all
standard Object Pascal types, property evaluation, watchpoints, and more.
27 automated integration tests cover the supported feature set.

Known limitations at this stage:
  - Linux x86_64 only (Windows and macOS planned)
  - Variant records, Variant type, and generics not yet covered


FEEDBACK

This is an experimental release — feedback, bug reports, and testing across
different codebases are very welcome.

  - OPDF / PDR issues: https://github.com/graemeg/opdebugger/issues
  - FPC compiler issues: https://gitlab.com/freepascal.org/fpc/source/-/work_items

Regards,
Graeme

[440bx]

I haven't tried it but, I'll ask what has become my "standard curiosity" question when I see an interesting project that is not trivial, did you use A.I to produce, design and implement OPDF and PDR ?

Thank you in advance and, I won't elicit further elaboration no matter what your answer is.

[Fred vS]

Congratulations @Graeme, always ready to keep FPC off the beaten track.
Can OPDF be used with FPC 3.2.2/3.2.4 (-ootb) as well?

[Graeme]

I haven't tried it but, I'll ask what has become my "standard curiosity" question when I see an interesting project that is not trivial, did you use A.I to produce, design and implement OPDF and PDR ?

I use AI for writing documentation, release notes, extracting change logs and help with proof-reading my design documentation. The boring or labour intensive stuff. I leave the coding for me, because that is what I enjoy doing.

OPDF is something I started 10 years ago, though I did take a 5 year break from Object Pascal to advance my career. The current version is my 3rd implementation, after a huge amount of research and trial-and-error. I don't use AI for Vibe Coding - I simply don't believe AI is ready for that, except for small utilities or prototyping.

[440bx]

I use AI for writing documentation, release notes, extracting change logs and help with proof-reading my design documentation. The boring or labour intensive stuff. I leave the coding for me, because that is what I enjoy doing.

OPDF is something I started 10 years ago, though I did take a 5 year break from Object Pascal to advance my career. The current version is my 3rd implementation, after a huge amount of research and trial-and-error. I don't use AI for Vibe Coding - I simply don't believe AI is ready for that, except for small utilities or prototyping.

Thank you Graeme, I appreciate the reply to my off-topic inquiry.

[Graeme]

Can OPDF be used with FPC 3.2.2/3.2.4 (-ootb) as well?

No.

That, combined with how slowly FPC releases are issued, is exactly why I decided to create an "unofficial release". I wanted to ensure that those who want to try it can do so without having to manually clone and build the FPC "main" branch themselves. If FPC were released more regularly, such unofficial releases would not be necessary.

OPDF consists of three parts:

1. Compiler support: The compiler must be able to generate OPDF data within the binaries it creates.

2. The OPDF debug engine: I created a standalone debug engine using the Hexagonal design pattern, so it is not tied to any specific IDE or tool.

3. The 'pdr' (client) reference implementation: A command-line debugger that allows developers to experience what the new format can do and demonstrates how the debug engine is utilised. It is a thin 500 lines-of-code layer, simply calling into the debug engine and displaying results.

I am also currently building a lightweight IDE within the fpGUI project, which serves as a second "client" to the OPDF debug engine. This showcases how easily it can be integrated into any IDE or editor. However, to fully test the debug support in this IDE, FPC 3.3.1 with OPDF support is required—hence my unofficial compiler release.

Since Windows and macOS support for OPDF are planned, and because FPC is lagging behind on official releases, I will be providing more unofficial releases for those platforms in the near future.

[440bx]

Before anything, I want to make it clear that I welcome the addition of a debug format that makes it possible for a debugger to handle all the features FPC implements.

I presume you chose to go with a different format instead of extending DWARF because DWARF extensions would probably need to be agreed upon with current FPC developers which could potentially greatly affect the amount of time in getting something done.  If the reason was some other, please feel free to point it out.

The point of this post is that, I find it quite discouraging that yours is now the second "unofficial" FPC branch in existence due to FPC's slow progress (release schedule.)  Effectively, there are now, at least, three FPC branches going in their own direction.  It would really be great if all that time, effort and talent could be focused on making one branch of FPC ever better.   Important: this is NOT a negative remark on you and/or @Fibonacci, just a remark on the resulting situation which, IMO, is highly undesirable.

All that said, the only "solution" I can think of is for FPC to have a shorter and more consistent release schedule and, at this time, I really have no idea how that may be accomplished.

[Fred vS]

Effectively, there are now, at least, three FPC branches going in their own direction...

Huh, there is also this one. 

[440bx]

Effectively, there are now, at least, three FPC branches going in their own direction...

Huh, there is also this one. 

I didn't know about your branch... add 1 to the count

[Graeme]

I presume you chose to go with a different format instead of extending DWARF because DWARF extensions would probably need to be agreed upon with current FPC developers...

No, not quite. Please see the my analysis document for full details: https://github.com/graemeg/opdebugger/blob/master/docs/analysis.adoc

That 'docs' directory contains more documents and information too.

All that said, the only "solution" I can think of is for FPC to have a shorter and more consistent release schedule and, at this time, I really have no idea how that may be accomplished.

I have recently raised this community fragmentation concern with the FPC developers in the fpc-devel mailing list. But it seems it's already worse than I expected. Sadly. :-(

[Martin_fr]

Out of curiosity:
https://github.com/graemeg/opdebugger/blob/master/docs/analysis.adoc#disadvantages-1

4.2. Option 2: Invest in fpdebug Project
...
Maintenance burden: Requires maintaining entire debugger stack

How does OPD solve that? Should OPD not eventually have a full "debugger stack"? (whatever you define to be part of it)?

Or is that an elaborate description of "needs to support handling the base level complexity of DWARF" ? => base level: For comparison with OPD, it only needs to support what FPC uses. Every other part of DWARF is not required for the comparison.

That said, this base-level is more involved as OPD likely is.

[Martin_fr]

Since I don't currently have the time to go through the sources, I ask here.

IIRC you expressed "concerns" about "DWARF expressions" (a rather flexible, but also involved way of describing locations and data).

How to you solve e.g. the following (if it happens / or has happened in the past)

Changes of internal offsets. E.g. in the "string header" (I recall there was some align changes, not sure if they affected the length, but then other fields...). So the location of those fields may change in future.

How is that location specified in OPD?
- Or do you add a new "string type" when it changes, and the debugger has hardcode info?
- Or is only the "current" supported? (Can OPD debug exes compiled with different future fpc versions, and keep compatible with all starting from now?)

Full disclosure: Because DWARF does not describe the string-encoding field, FpDebug has such hard-code info. (Sadly, but ...). However it will be able to deal with offset changes, in released versions (i.e. ONE hardcode constant per FPC version // 3.3.1 can only have current)

[Martin_fr]

VSCode integration

Btw, there is a DAB version of FpDebug (by Joost). I have not tested it, and I do not know its maintenance state.

Joost also did an IDE add on to run DAB backends in the IDE. But I am not sure that is currently up to date. And also, when I last checked it used its own frontend (its own set of watches/locals windows). So I don't know how it compares to the latest changes in the IDE's own frontend.

Sorry, slightly off topic. Just for completion of the list.

[440bx]

No, not quite. Please see the my analysis document for full details: https://github.com/graemeg/opdebugger/blob/master/docs/analysis.adoc

That 'docs' directory contains more documents and information too.

Thank you Graeme.  I read the documentation in the link you provided.  I didn't read anything else (yet).

I have a question at this time.  Does the _current_ OPDF support block scoped inline variables (e.g, by using a combination of OPDF elements to describe the feature or, would some specific element need to be added to specifically describe that feature) ? ... the purpose of my asking the question is to have some idea of how OPDF represents some language features.  Entirely a matter of curiosity at this time and, not really wanting to read the entire's project documentation yet to figure out what the answer is.

[Graeme]

Since I don't currently have the time to go through the sources, I ask here.

Hi Martin,
                                                                                                                                 
Today, OPDF is in exactly the same boat as fpdebug.
TDefAnsiString carries only a TypeID and a name, and pdr's string evaluator
hardcodes "length at pointer-8, refcount at pointer-4, data at pointer+0". If
FPC changes the header tomorrow, my debugger breaks until rebuilt — same
trade-off you've already worked through.
                                                                                                                                 
The thing your question changes for me, and the reason it's well-timed: OPDF is
still in pull-request state with the FPC team. The spec is genuinely still
malleable, with no deployed binaries to stay compatible with. So if extending
TDefAnsiString to carry header field offsets — LengthOffset, RefCountOffset,
CodePageOffset, ElementSizeOffset — is the right call, I can put them straight
into v1 of the record before the PR lands. The FPC writer (best placed to know
the exact offsets) emits them at compile time, and any consumer reads them at
runtime with no recompile and no per-FPC constant table.

You've raised a genuinely good point here, and I think extending the record
structure before OPDF merges into FPC is the right long-term strategy. Every
offset and layout assumption I can move out of the debugger and into the format
itself is one less hardcoded constant the debug engine has to carry — and one
less thing that has to be patched when FPC's internals shift. The whole spirit
of OPDF is "let the compiler describe what it knows", and string headers fit
that brief as well as anything. I'd rather take the time to get this in now than
ship a v1 that quietly relies on the same hardcoded knowledge fpdebug already
has to maintain.

To answer the cross-version question explicitly: an OPDF binary produced today
by FPC 3.3.1 will remain readable by all future pdr versions — the format is
versioned, every record carries an explicit RecSize, and unknown records are
skipped cleanly. What is not guaranteed yet is that pdr will interpret an
AnsiString whose offsets have shifted underneath it. The descriptor would close
that gap.                     
                                   
Two other limitations worth flagging in the same spirit: OPDF doesn't model
register-allocated variables yet, and it doesn't have anything like .eh_frame —
it leans on RBP frame chains and breaks the moment the optimiser drops them.
Both are known gaps. The proposal isn't "OPDF replaces DWARF"; the two sit
happily in the same binary in separate ELF sections. Anyone who needs GDB
compatibility keeps using -gw; anyone who wants first-class Object Pascal
debugging adds -gO. No pressure to deprecate anything.
                                   
On a side note: since you'll likely browse the docs at some point: the early
analysis and design documents in the repo predate the current implementation and
may read as slightly dated in places. I've kept them in deliberately to show the
rationale and the design evolution — this is actually my third attempt at
getting the format right, and I think there's value in leaving the trail visible
rather than rewriting history to look planned.                   
 
Cheers,                                                                                                                         
  Graeme