Lazarus 3.9.9 Variable Watches Fail (But Debugger Tooltips Work!)

[Martin_fr]

Step through one line of code, and the entire window loses its entire contents (just the column headers, with a gray empty region below).

So that could be debugger, or GUI....

The first question at this time is, if the problem is
- the target (the project being debugged) is build with FPC 3.3.1
- the IDE itself being build with FPC 3.3.1

FPC 3.3.1 has a lot of new code and changes. It is possible that it does not translate the sources correctly.

E.g. even fpc 3.2.2 has bugs, and one of them means that fpdebug build for 64bit X86 for Windows with -O2 or higher => will not work because of wrongly generated machine code. (Well at least some revisions of the IDE, as it depends on surrounding code)

So the same could apply to you building the IDE with 3.3.1. Hence, if you don't by default build with -O- => could you test building you IDE with -O- ?

----
The fact that the alpha vs beta makes no difference is interesting. Then it would seem that it does not matter how the watch value is provided (goes along with the fact it works in hints).

But that would mean that the problem is somewhere in the watch window itself. And that is not depending on the underlaying architecture. So it should happen for other people too... (on other OS)...

Can you test:
  What happens if you add a new watch while being paused?

[Martin_fr]

And also, when you step, do the watches in the watch window temporary show "evaluating"? (or at least a hint of a flicker that the window gets updated?)

Add a lot of watches (20 or more), so it is easier to spot.

[msintle]

Just getting back into this after a prolonged absence due to work demands.

Is everything on this thread current, should I just collect the data you had requested earlier, or has anything changed with the new Lazarus 3.0 release?

I haven't yet upgraded (fears of breaking my build environment abound) so I don't have an inside look into that version yet.

[Martin_fr]

Sorry, lots of other things that I had on my mind since.

Trying to recap.

- So pretty much any app produces the error?
- Type of data does not matter? Simple integer will do.
- It will show in the hint.
- It will show an (error|nothing|...) in the watches window? (IIRC "invalid")?

This is true
- if this is the only watch?
- even if the watch is added after the app got paused?

This is true for
- local vars too?
- global vars?
- fields/other?
Do the locals show in the local window?

What if you just enter the digit "1" (without quotes) as a watch? It should eval to 1.
If so, enter a "1" above and another below the failing watch (do not reorder the watches, clear the watches, and enter them in that order).


And yes, a logfile is probably best.

[msintle]

Sorry, lots of other things that I had on my mind since.

It is I who owes you the apology.

- So pretty much any app produces the error?

Yes. I just created a new app with this code:

Code: [Select]

procedure TForm1.Button1Click(Sender: TObject);
var
  i: Integer;
begin
  for i := 1 to 11 do
    ListBox1.Items.Add(IntToStr(i)); // set breakpoint here
end;

With just the button and the list box on the form. And indeed, after i = 1 showing properly the first time I stopped at the breakpoint only, sure enough, the issue reproduced instantly the second time I stopped there.

- Type of data does not matter? Simple integer will do.

Yes, per the above sample too.

- It will show in the hint.

Indeed.

- It will show an (error|nothing|...) in the watches window? (IIRC "invalid")?

Absolutely, just says <invalid> in the Value field, with the square brackets.

This is true
- if this is the only watch?
- even if the watch is added after the app got paused?

Yes even then, but not if only then. Doesn't matter!

This is true for
- local vars too?
- global vars?
- fields/other?
Do the locals show in the local window?

Locals vs. globals don't matter.
I just added a global variable j to the sample above and set it to i on each iteration of the loop.

Interesting observation: The first iteration right after adding j, it worked fine. The second iteration and all subsequent iterations, it became invalid again.

The interesting part specifically being, this "state" persists between runs. So when I restart the app after breaking the debug session, everything shows as <invalid> again. Including j, which had worked for the first time only right after I had added it.

What if you just enter the digit "1" (without quotes) as a watch? It should eval to 1.

Great idea! Guess what - worked only the first iteration right after I added it. Stepping through code again, it failed again. Just like j, it never worked even after restarting the debug session.

If so, enter a "1" above and another below the failing watch (do not reorder the watches, clear the watches, and enter them in that order).

Did that "offline" while the program was in a terminated state. No change.

Closed and restarted IDE. Again, no change, everything invalid - isn't that crazy? Not even valid even the first time stopping at the breakpoint!

And yes, a logfile is probably best.

That's the only step I haven't done yet. Did I need to recompile the IDE or similar for that?

[Martin_fr]

ok, few interesting observations.

1) As the digit 1 also fails => It isn't about getting data from the debugged exe. 1 is just a number, and will be returned as such (it still is evaluated by the debugger, but only the FpDebug part, not the lldb side)

2) This kind of "error is stuck" means something must go wrong globally.

There could be an exception of some kind. It might not be reported as it happens in a different thread.
Or there could be some memory incorrectly overwritten (dangling pointer or complier issue).

I understand you are using FPC 3.3.1 ? That could have bugs (and those could include wrong code generation / has happened before)
So
- At the very least compile your entire IDE with -O-  no optimizations, minimize the risk.
- If you have at hand, try 3.2.2 or 3.2.3  (3.2.2 also may have issues, so also -O- or -O1 / 3.2.3 should be good)

Just for info, I spent (meanwhile a year or longer ago) more than 4 weeks searching a bug because FpDebug kept failing. Turned out to be a bug in the fpc 3.2.2 code generator. The generated machine-code simply was wrong.
3.3.1 has a lot of new optimizer additions. So this is a possibility.

You can build an IDE with 3.2.3, and then use 3.3.1 to compile your apps. It is only about what was used to build the IDE.

[msintle]

OK, I'll fire up a VM to create a new environment and get you the datapoints you need from that environment.

Would you mind summarizing exactly what I should be testing and the steps to run those tests?

Ex: Logging, rebuilding IDE, etc.

Whenever you have the time, of course.

[Martin_fr]

Lets try the following 2 steps first

1) Log file
You can start with a logfile with no extra preparation.

Just run the IDE with (not sure about the exact path, but something like)
./lazarus.app/MacOS/Content/lazarus  --debug-log=file  --debug-enable=DBG_VERBOSE,DBG_WARNINGS,DBG_ERRORS
DBG_STATE,DBG_EVENTS,DBGMI_QUEUE_DEBUG,DBG_FPDEBUG_VERBOSE,FPDBG_COMMANDS,FPDBG_THREADS,FPDBG_QUEUE,DBG_DATA_MONITORS,DBG_THREAD_AND_FRAME

And send that log.
I can then decide if further logs are needed.


2) Compile the IDE with -O-
You can try that with fpc 3.3.1 or whatever you already have (for now)

You can just rebuild the IDE from the tools menu "Configure build lazarus" and specify
 -O-  -gh  -Sa  -CRiot

This will slow it down a bit.

Copy your existing exe first (the exe is outside the app bundle ! / next to it / app bundle just contains a sym link)

[msintle]

Here's the log file for the first step.

We do still need the second step, right?

[msintle]

Trying #2 produced this error when I clicked Build:

Error: Illegal parameter: -O-

[Martin_fr]

-O-
minus  uppercase-letter-O  minus

https://www.freepascal.org/docs-html/user/userap1.html
Strange. It definitely works here. I don't see how this could be OS/Target specific. It simply says: Don't optimize. That should work on all targets.

Maybe something to run past the FPC team....

But never mind, after looking at the log, it wont (most likely) be relevant for this case.


Currently I assume that some output format of lldb may have changed.

Even for a watch like "1" the debugger tries to get the current pause location. As this is needed in many cases. If that fails, it also fails watches that don't need it.

While the log has not enough info to trace it all the way, it does give me a start point.

Code: Text  [Select][+][-]

1. GetInfoContextForContext ADDR NOT FOUND for 1, 0

Could you please run the log again, but this time with


./lazarus.app/MacOS/Content/lazarus  --debug-log=file  --debug-enable=DBG_VERBOSE,DBG_WARNINGS,DBG_ERRORS
DBG_STATE,DBG_EVENTS,DBGMI_QUEUE_DEBUG,DBG_FPDEBUG_VERBOSE,FPDBG_COMMANDS,FPDBG_THREADS,FPDBG_QUEUE,DBG_DATA_MONITORS,DBG_THREAD_AND_FRAME,DBG_CMD_ECHO,DBG_CMD_ECHO_FULL

Also at the same time please do the following
- Before you start the debugger, open the "Debug output" window from the menu "View" > "IDE Internals" > "Debug Output"
- Then debug, the window will fill up
- Once the debugging is done, copy the content of that window, and attach it here

Please alse (re-) confirm: When hovering the mouse and getting a hint on a variable, then the correct result is returned. This is for the same variable as in the watches window.

I know you said that before. Just double checking.
There is one tiny differences in the debugger code between those 2 cases.


If this is a described, then include this in the log as follows.
- first let the debugger fail on the watches window
- only then / only after the watches failed: Hover and get a hint
  (Don't step or run)
- kill the debugger: menu Run > Reset debugger
  Don't do anything else after the hint => so I can find it as the last value-eval in the log.


Btw which exact CPU (M1, M2, M3) do you have? (afaik those are the ones currently avail?)

[msintle]

So once again forgive me for the extended delay in getting back to you on this.

I've attached the logs you requested and I've followed your instructions to the letter.

This is on an M2 CPU, if that makes a difference.

To double-confirm as you requested:

0) Yes, as I copy/pasted the flags from here, I definitely used uppercase letter O (not a zero).
1) Hovering always returns correct results.
2) The debugger, as usual, "remembered" its past failure and failed instantly upon reaching the breakpoint.
[ To clarify, I did not delete the original testing project, despite the extensive delays I've had in getting back to you]

And a question of my own:

I find it interesting that there's no comma between the debugger command line flags you specified - this is as intended, right? That's where the new line has been introduced (maybe by the forum), most probably it's nothing but just double checking.

[Martin_fr]

I am still looking, but in the meantime.

1) Please try with this patch

Code: Text  [Select][+][-]

1. diff --git a/components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas b/components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas
2. index 78d21029f5..2a4fa9c37a 100644
3. --- a/components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas
4. +++ b/components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas
5. @@ -1554,6 +1554,9 @@ function TFpLldbDebugger.GetLocationForContext(AThreadId, AStackFrame: Integer):
6.    r := Registers.CurrentRegistersList[AThreadId, AStackFrame];
7.    if (r <> nil) and (r.DataValidity = ddsValid) then
8.      try
9. +      if FDwarfInfo.TargetInfo.machineType in [mtARM, mtARM64] then
10. +        v := r.EntriesByName['PC'].ValueObjFormat[rdDefault].Value[rdDefault]
11. +      else
12.        if TargetWidth = 64 then
13.          v := r.EntriesByName['RIP'].ValueObjFormat[rdDefault].Value[rdDefault]
14.        else
15.  

2) Does it affect all stackframes?

If you open the stackwindow and select any of the caller frames.
Ideally you do that in some code where the callers have debug info, e.g if you make calls from your code to other parts of your code).

Can you then get watches for the caller to work?

[Martin_fr]

OK, so the findings got odd enough, and my curiosity overcame my "Mac-aversion" -ish.

It seems that the watches does not work, if the stack Window is closed. If you keep it open, then Watches will (most likely) work.

And it seems the patch makes it work, even if the stack window is closed.

I saw the register PC vs RIP before, but since that code worked for everybody else... I thought it couldn't be. Of course tiny difference in what everybody else just happens to do....

[msintle]

OK, so the findings got odd enough, and my curiosity overcame my "Mac-aversion" -ish.

A friend of mine always says "welcome to the dark side" once one onboards Macs. I have always wondered why! Just asked him, but he has ignored the question before.

It seems that the watches does not work, if the stack Window is closed. If you keep it open, then Watches will (most likely) work.

And it seems the patch makes it work, even if the stack window is closed.

I saw the register PC vs RIP before, but since that code worked for everybody else... I thought it couldn't be. Of course tiny difference in what everybody else just happens to do....

That's the Call Stack window, right?

Didn't work for me without the patch.