Author Topic: FpDebug: Bugs and API gaps, with proposed fixes  (Read 2614 times)

MattBradford

  • New Member
  • *
  • Posts: 34
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #30 on: September 10, 2026, 02:14:39 pm »
Correcting myself first, because you were right and I told you otherwise.

FindNamedProcSymbol does have a real caller, and it is mine

You said "I thought you had it added to your debugger already". You were right. My reply #28 said the temporary probe was "the only thing that runs FindNamedProcSymbol at all", and that is wrong.

The out-of-tree consumer I work with resolves every by-name breakpoint through it: one call with [psfDwarfName], and on a miss a second with [psfLinkTableSym], to know which namespace answered. That is production code in its session layer, not scaffolding, and it ran as recently as yesterday - a breakpoint set by function name, in a run made for an unrelated purpose.

How I got it wrong: I was thinking about the fpd probe I have been quoting numbers from, and wrote as if that were the whole picture, forgetting the consumer the by-name lookup was added for in the first place. It is the one caller I should not have overlooked.

What survives from that paragraph is narrower and still true: there is no testcase, and nothing inside the Lazarus tree calls it. So the path is exercised daily by one out-of-tree consumer and by nothing you can run.

The commit that landed is not the hash you named, and it does not matter - but it is worth saying once

b85b31d7a1 is not in main. What is in main is b0d3c667a3: same change, rebased onto a newer base when you pushed. You will know that, having done it; I mention it only so that anyone reading this thread later looks up a hash that exists in the history.

I checked rather than assumed that it is the same thing I measured:

git range-diff reports the two commits as identical, and fpdbgdwarf.pas is the same blob at both. Exactly one commit touched components/fpdebug between my test base and current main, and it is that one. So the five rows I posted in reply #26 transfer to main unchanged. b0d3c667a3 is the hash to cite.

The testsuite - starting now

Understood on both counts: they are mine, and what you ran was the existing unmodified suite against your change rather than a new case. Nothing to build around, so I will build from the shape agreed in reply #15 plus your two extra cases.

The one thing I could not settle out of tree is the scoped-enum fixture. My probe asks the generic symbol lookup, where fsfOnlySubroutines is not in play, so it never reaches the proc path the fixture exists to test - and on that lookup the enumerators are not findable at all, which means the collision I wanted to demonstrate never actually took place. The testcase is the right instrument for it and I will let it answer there rather than guess here.

Threads - one place your two answers meet

On the worker pool you said: if you create a pool you need to shut it down, and if you do not create one it works as described. On FpDebug you said it "has some code that starts threads itself (finding symbols by name / hashing names)".

Those meet on my consumer, and I think the second one moves the first: it never creates a queue, and the path it drives hardest is by-name symbol lookup. So a consumer that has done nothing threaded can still end up with FpDebug's own threads, which is exactly the internal usage you say should be addressed. If that is right, then "if you don't create one" is not yet a state a consumer can choose.

I am not asking for work on this and I am not filing anything. I raise it only because it makes the factory idea more attractive than it looks: passing a no-thread queue would give a single-threaded consumer a way to say so, rather than relying on not touching an API it already is not touching.

Happy for it to become its own topic whenever you want it to.

On timing

Noted on the week away. Nothing I have needs you before then - D4 stays parked until fixes-5 branches, and the testsuite work does not need you while it is being written.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12760
  • Debugger - SynEdit - and more
    • wiki
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #31 on: September 10, 2026, 02:23:46 pm »
Quote
The commit that landed is not the hash you named
Yes, rebased ;)  One of my favourite git features. MR get merged (after rebase, merge with --no-ff). My own commits get applied directly.



Quote
. If that is right, then "if you don't create one" is not yet a state a consumer can choose.
True.


MattBradford

  • New Member
  • *
  • Posts: 34
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #32 on: September 11, 2026, 11:21:51 am »
The TestFpDebugApi testcase is filed as a Draft: !765.

Four commits: the fixtures into WatchesScopePrg, the test unit, and two corrections to those that I found while writing the MR description. Squash them down if you prefer two. No new exe. 37 assertions - absent name, plain procedure, case on both the DWARF side and the link-table side, the own-compilation-unit collision, the round trip, the two namespaces on FPC_BREAK_ERROR, and the scoped-enum collision.

First, a correction to something I repeated from you

I wrote that LazDebuggerFp does not call FindNamedProcSymbol, taking it from your "neither testcase, nor running in a debugger / LazDebuggerFp does not call it". The testcase half is right. The other half is not, and I should have checked it instead of repeating it. Three callers:

Code: [Select]
fpdebugdebuggerbase.pas      [psfLinkTableSym]  RTL linker names
fpdebugvalueconvertors.pas   [psfLinkTableSym]  a mangled name
fpdebugconvdebugforjson.pas  no flags           ALL namespaces

The third is the one this testcase is about, and the comment above it - which I put there in c842365ef9 - says "All namespaces, deliberately, and the only caller in the tree that gets them". So the DWARF-side behaviour these assertions pin has exactly one in-tree consumer, and it is the one taking a user-supplied source-level name. That is a better reason for the test to exist than the one I started with.

Measured on all nine cells I have here: fpc 3.3.1 at stDwarf, stDwarfSet, stDwarf3, stDwarf4, stDwarf5, and 3.2.2 at the first four. Green in all nine, and then every cell re-run with one expectation deliberately wrong, because nine greens do not distinguish a cell that ran 37 assertions from one that ran none. All nine went red naming the assertion, so all 37 execute everywhere. win64 only.

What the two namespaces do with case

As I noted it down from your review: from Procedure FooBar, findable as FOOBAR or FooBar, not as fOobAR. If I took that down wrong, say so and I will re-test what you actually meant.

On the DWARF path it is found, at the same address as FooBar, under DWARF 2 - the version the case was stated under. The comparison there is CompareUtf8BothCase in fpdbgutil.pp, reached on the proc path through GoNamedChildEx and on the unit-name branch directly. It walks the stored name and accepts each character against either the upper or the lower form of the requested one, so for an ASCII identifier I cannot see how a casing would miss. That is the measurement plus a reading of the comparison, not a proof - there is a name-hash gate in front of it that I have not read. I am not suggesting any change; case-insensitivity is the right behaviour for Pascal.

On the link table you are exactly right. TfpSymbolList.GetInfo is two steps:

Code: [Select]
1. FNameDict.TryGetValue(UpperCase(AName), i)   case-INsensitive
2. walk the chain for p^.Name = AName           exact

psfIgnoreCase relaxes step 2 and never touches step 1. So fpc_break_error with [psfLinkTableSym] misses, and the same string with psfIgnoreCase added hits.

So a caller that gets the case wrong is served by one namespace and refused by the other. The test asserts both halves, because I could not find that written down anywhere, and an exact-match change on the DWARF side would look harmless while breaking it.

One more thing falls out of step 2 being the only part relaxed: AFoundName comes from p^.Name in both modes, so the link table reports the spelling it holds and not the caller's. That is the same property your '' change gave the DWARF side, and the test now pins it on both.

The other correction, since it bears on your tree and not just mine

One fixture statement was outside the never-true guard, and it was the statement immediately after TEST_BREAKPOINT=Prg - so it is the line a stepping test lands on next. Everything is inside the guard now, and nothing the fixtures add executes. The same commit fixes the comment on the FooBar fixture, which had asserted the fOobAR case as fact.

Checking the first of those meant checking something the test cannot see. The scoped-enum fixture is a collision, and if the enum stopped reaching the DWARF the assertion would still pass - there would simply be nothing left to compete with the procedure. So I probed the built testee object directly for the enumerator, with the procedure names as positive controls and a bogus name as a negative control. Present before and after, and the object is the same size, so the debug info did not move.

What I have not run

Any other suite. WatchesScopePrg is in lazdebugtestbase and shared - LazDebuggerFp's own TestWatches uses it, and so does the LazDebuggerFpLldb suite, and I have re-run neither. Nothing the fixtures add executes now, so I do not expect a difference, but that is read rather than measured. If you would rather they sat somewhere only LazDebuggerFp compiles, I will move them.

Three things I left out on purpose, because they are yours

The mangled-name assertion. You wanted the test to use the debugger's own mangling code so a new FPC breaking it shows up. That code is on TFpValueDwarfFreePascalSubroutine - a value, reached by evaluating an expression, not a TFpSymbol reached by a lookup. So your way pulls the watch machinery into a test about symbol lookup, and a local copy of the formula tests FPC but not the helper, which I think is the opposite of what you wanted it for. I would rather ask than pick.

The unit-name half of the Sym.Name assertion. It is untestable in this testapp: every declared unit name here already matches its file basename exactly, casing included, so the assertion would pass while discriminating nothing. A real one needs a mismatched filename, which means an extra source file - still no new exe, so I think your point about each extra exe multiplying is met, but it is your tree.

Whether a testapp change should ship a regenerated sources.res. I have not touched it. Changing the testapp leaves that generated file stale, which is harmless for you since the .rc regenerates at compile time. I have no windres and regenerated locally with fpcres, which the comment at the top of testcommonsources.pas already anticipates - so it is only a question about the committed copy.

It is a Draft because the first two would change the diff. Say which way you want them and I will add them before marking it ready. Nothing else is asked, and nothing here needs you before you go.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12760
  • Debugger - SynEdit - and more
    • wiki
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #33 on: September 12, 2026, 08:38:14 pm »
will check the latest when I am back in a fortnight. / to busy now.

MattBradford

  • New Member
  • *
  • Posts: 34
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #34 on: October 01, 2026, 06:36:01 am »
Thanks for merging !765. Here is a batch of things I found since then. Items 2, 3 and most of 6 are in fpd, the console tool, rather than in the IDE debugger. Item 4 is in FpDebug itself. Item 5 is D4. For each one that has a fix: would you like an MR, or would you rather take it in yourself? None of them is urgent.

1. The follow-up I owed from #32: the other suites on WatchesScopePrg

I said then that I had not re-run the suites that share WatchesScopePrg. I have now run LazDebuggerFp's two TTestWatches tests that load it, TestWatchesScope and TestClassMangled. Each ran in all nine cells (fpc 3.3.1 at stDwarf, stDwarfSet, stDwarf3/4/5; 3.2.2 at the first four), once at the !765 merge and once at its parent. Both pass everywhere. In every cell I also ran them with one deliberately failing check, to count the checks that actually execute. The count is the same before and after the merge: 5912 or 5902 for TestWatchesScope, depending on DWARF version, and 13 for TestClassMangled. win64 only.

One trap I hit on the way, in case it helps anyone: an fpclist.txt entry without a version= line leaves Compiler.Version at -1. TestClassMangled then exits at its "Compiler.Version < 030000" check and passes in 0.000s, having run nothing.

I could not run the LazDebuggerFpLldb suite, as there is no lldb on this machine.

2. fpd: eval of any record or array raises an access violation

At current main, eval in fpd prints scalars fine, but a record, a static array, a dereferenced record pointer or :flatten() all raise EAccessViolation. I built fpd with line info and dumped the exception stack:

Code: [Select]
DOSTRUCTURE         fppascalbuilder.pas:1247   (records)
DOARRAY             fppascalbuilder.pas:1379   (arrays)
INTERNALPRINTVALUE  fppascalbuilder.pas:1595 / 1599
PRINTVALUE          fppascalbuilder.pas:1620
HandleEval          fpdcommand.pas:588

Both faulting lines read Context.MemManager. HandleEval creates a TFpPascalPrettyPrinter and sets AddressSize, but never sets Context. fpdserver's eval in debugthreadcommand.pas has the same shape and does set it (line 703), as LazDebuggerFp does (fpdebugdebuggerworkthreads.pas:1329, fpdebugdebugger.pas:2047). With this one line added, all of the above print correctly and nothing is raised:

Code: [Select]
--- a/components/fpdebug/app/fpd/fpdcommand.pas
+++ b/components/fpdebug/app/fpd/fpdcommand.pas
@@ -578,6 +578,7 @@ begin
       APrettyPrinter := TFpPascalPrettyPrinter.Create(4);
       try
         APrettyPrinter.AddressSize:=AContext.SizeOfAddress;
+        APrettyPrinter.Context := AContext.LocationContext;
         if APrettyPrinter.PrintValue(AVal, APasExpr.ResultValue) then
           begin
           Writeln(AVal)

3. fpd: access violation on exit, even after only "quit"

Stock fpd exited with 0xC0000005 in every run I made, including sessions whose only input is quit, and at every commit I tried back to August. Under a debugger it stops in TObject.Free, called from TFPDLoop.Destroy (fpdloop.pas:292), from fpdloop's finalization. Lines 289 and 292 both free FMemManager, and FMemModel is never freed. The second Free came in together with FMemModel in f9e55734c6. fpdserver's TFpDebugThread.Destroy frees the model at the matching place.

Code: [Select]
--- a/components/fpdebug/app/fpd/fpdloop.pas
+++ b/components/fpdebug/app/fpd/fpdloop.pas
@@ -289,7 +289,7 @@ begin
   FMemManager.Free;
   FMemReader.Free;
   FMemConvertor.Free;
-  FMemManager.Free;
+  FMemModel.Free;
   GController.Free;
   inherited Destroy;
 end;

With that line, a quit-only session exits 0 in 5 runs out of 5 (stock: an AV in 5 out of 5), and heaptrc reports no unfreed blocks.

4. What is left at exit after that: the worker threads, from #30/#31

With item 3 fixed, a session that has run a program still often ends badly: runtime error 204, an AV, or a hang that never exits. heaptrc reports "pointer ... does not point to valid memory block". Most of the stacks are on a worker thread at fpdbgutil.pp:1121, the WorkerThread-Exit debugln, inside DbgS/DbgSTime. One was on the main thread, in fpdbgutil's finalization, freeing the global worker queue (:1434).

20 debug sessions per build:
  • Items 2 and 3 applied, heaptrc on as fpd's project has it: 13 bad (10 crashes, 3 hangs).
  • The same, plus FpDbgGlobalWorkerQueue.TerminateAllThreads(True) as the first line of TFPDLoop.Destroy: 0 crashes, 2 hangs.
  • Item 3 only, heaptrc off: 0 bad.

So the worker threads are still running when unit finalization starts, and with heaptrc that goes wrong. The out-of-tree debug server I mentioned in #30 hit the same frame in August. It faulted at exit in 6 of 16 runs, from only launching and killing a program, in a heaptrc build made for the investigation. It has called that same TerminateAllThreads(True) before shutting down ever since, with 0 faults in 44 runs. I don't know why heaptrc turns this into errors, or where my two remaining hangs come from.

A question, not a proposal: should a host like fpd join the global queue itself before shutdown, or is that something FpDebug should do? If it is the host's job, fpd needs that line as well.

5. D4 (address 0): now that fixes_5 has branched

As I understood your reply in August, there was to be no big rework until fixes-5 branched, and fixes_5 branched on 27 September. If this is a good time, two questions from August are still open:
  • Does the move also cover TFpValueDwarfPointer.GetDerefAddress? It has its own IsNilLoc shortcut (fpdbgdwarf.pas:2859) and never reaches MemManager.
  • Do you want the diagnostic-only part first (a real error on that nil branch instead of "Internal dereference error"), or together with the move?
I won't start on it until you say.

6. Smaller things, one line each
  • fpd never frees a breakpoint unless it is removed with "break -d". After any session that set one, heaptrc lists three unfreed blocks: the TFpInternalBreakpointAtFileLine from HandleBreak, a string from the GetPart call in HandleBreak, and an address list from UpdateForLibraryLoaded. Finalize frees BreakPointIdMap but not the breakpoints in it.
  • TBreakPointIdMap.DoBreakPointCompare in fpdcommand.pas returns PPointer(Key1)^ - PPointer(Key1)^, so it is always 0. The compiler's own hint points at it ("Parameter Key2 not used"). Nothing in fpd does a data-side lookup, so I have not seen it cause anything.
  • LazDebFpTest.lpr: switching on NOGUI swaps the runner, but TestDbgControlForm stays in the uses clause, and its initialization ties the test gates to the form. Excluding that unit under NOGUI as well gave me a working console run.
  • testcommonsources.pas offers fpcres as an alternative to windres, but fpcres rejects sources.rc's unquoted dotted names ("syntax error at 1:20"). With the names in quotes, which windres also accepts, it compiles. You regenerate sources.res yourself, so this is only a note.

Everything above is win64 only. Unless the item says otherwise, it was measured at current main (11a5f5d500) with fpc 3.3.1.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 12760
  • Debugger - SynEdit - and more
    • wiki
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #35 on: October 07, 2026, 12:18:07 pm »
Bit busy right now / squeezing an answer in / might be a bit before looking at any details

Quote
The follow-up I owed from #32: the other suites on WatchesScopePrg

Missing "fpc version": Yes should probably have a default.... Or maybe cause an error "config not readable"

LLDB: will be run on my site

Quote
fpd: eval of any record or array raises an access violation
Is indeed not maintained :( sadly...

I occasionally try to fix compilation. I haven't yet gotten a setup for testing it. Would be good to have. But "no time" (I am still behind running the fp-lldb, and can't even recall when I run tests for the fp-gdb and gdb / which also need to be fixed...)

Please report your fixes on the bug tracker. I can't start tracking open issues on the forum... (and if I owe an answer - on the forum - for more than 4 week => ping via pm please)

Quote
What is left at exit after that: the worker threads, from #30/#31

I haven't checked where that queue comes from (also is that fpd only, or affects your own debugger too?)

There are thread workers inside FpDebug (used in TDwarfCompilationUnit) => TFpThreadWorkerScanAll / TFpThreadWorkerComputeNameHashes.
Those may need work on the long run (but I don't know yet when it may happen). They should only be threaded, if there is a queue that was created by the outer application / the app that uses FpDebug. Otherwise they should not run in threads (not sure what they do now...)

So it may be that for now any app using FpDebug MUST create a thread queue.

If an app (including fpd) does that (willingly or forced), then the app must close it too.

- FpDebug itself does not manage that queue.
- FpDebug should at some point wait for its own running threads, if e.g. it gets destroyed

The latter should allow the outer app to ensure the internal objects are waited for. If that is currently not possible, then that part needs to be fixed inside FpDebug

It should be ok, since
Code: Pascal  [Select][+][-]
  1. destructor TDwarfCompilationUnit.Destroy;
  2. begin
  3.   FOwner.WorkQueue.RemoveItem(FComputeNameHashesWorker);
  4.   FOwner.WorkQueue.RemoveItem(FScanAllWorker);
  5.  

Quote
D4 (address 0): now that fixes_5 has branched

I don't think "GetDerefAddress" itself is an exception => I.e. it should also go via the mem-manager (or otherwise allow to skip that check).

Where I do think the check should remain (or should have a config of their own, with defaults to have the nil check)
- classes (TObject): (for now all, eventually if the CU is Pascal) since the language itself has "if myObj <> nil then"
  (may include RTTI reads)
- Dynamic array
- not sure about Pchar (see below)
- not sure: pointer to record

If I understand correctly the 0x00 address is a system/kernel/hardware issue. So it should be looked up either for its own value, or as pointer to non-Pascal data (but the above are pascal data).

Further, even if that was used as very normal memory, the first word/dword is used by the mem manager (except constant record or string, which is the "not sure" bits, if a pointer to such a constant exists).
A TObject and DynArray however can not have that value, as there date is always allocated dynamically.

If the above run on nil, they are therefore highly likely to produce trash.
(Well Pchar or record could also be used for viewing such kernel/hardware info...)


If TObject needs (or benefits) from a check in "GetDerefAddress" then a flag would be needed.

Quote
Smaller things, one line each

"FPD / breakpoint" => Issue with/without Patch or MR please. (see above, currently not a priority for me)

"break cmp" : fixed

"NOGUI" : create issue please / later

"testcommonsources": create issue please / will test whet I have time.





MattBradford

  • New Member
  • *
  • Posts: 34
Re: FpDebug: Bugs and API gaps, with proposed fixes
« Reply #36 on: October 11, 2026, 04:36:02 pm »
Thanks for squeezing that in. Moved to the bug tracker as asked:


Each one is self-contained, so nothing in this thread needs re-reading.

To your question on the worker threads ("fpd only, or your own debugger too?"): both. Same frame, same remedy. The details are in #42645.

Thanks also for running the LLDB suite, and for the D4 answer. Noted.

 

TinyPortal © 2005-2018