Problems with Debugger on OSX

[ahnz]

Hi,

*** SUCCESS ***

I think we have it working! - Unless you have any counter ideas and want me to try different things

I tried the patching, applying line by line but I think its become apparent that I don't have the previous revisions and things like FReaderErrors which are there in rev. 60899 (and maybe earlier) don't existing in my base v 2.0.0 install and so a recompile of the IDE failed. Maybe I need to learn how to keep upto date with these revisions rather than waiting for a new stable release such as 2.0.2.

As for the other line (it is the  2nd line in the stack trace)
components\lazdebuggers\lazdebuggerfplldb\fplldbdebugger.pas
line 1120 (yours may differ +/-50)

Code: Pascal  [Select][+][-]

1. procedure TFpLldbDebugger.DoState(const OldState: TDBGState);
2. ....
3.       FWatchEvalList.Clear;
4.     end;
5.   end;
6.   if (State = dsRun) and (FMemManager <> nil) then
7.     TFpLldbDbgMemCacheManagerSimple(FMemManager.CacheManager).Clear;
8. end;

Above already contains the changed line.

I did manage to get this done and hey presto, the barebones app compiles and runs with debug without issue

My own app, now compiles with both 32/64bit and runs with debug, but I'm getting a project raised exception class "EInvalidGraphic" with message Unknown picture format'. This is odd because the application works fine without complaint if run without debug.

I just need to now figure out how to use the debugger properly - any suggested links on good tutorials??

Thanks for all your help - much appreciated!!!!

[Martin_fr]

Code: Pascal  [Select][+][-]

1. if (State = dsRun) and (FMemManager <> nil) then

And with that line added, you can see locals, and inspect variables?

About the exception. The debugger will tell you about every exception when it is "raise"d. Even those, that are later caught and handled.
So try continue on the exception, maybe it gets handled.

You can add individual exceptions (by exception class) to an ignore list. But that is independent of they being handled, or if there is an except block or not.
To add them there is a checkbox in the dialog. To see those added, go to Tools > Option >Debugger > Language Exception.

[ahnz]

Now that I've had a chance to play with it, I can see that while the Debug stops at the break point and I can step through the app. I can not in fact see any local variables

Of the Debug Windows, only the BreakPoint, Register, Call Stack and Thread windows give any information.

When trying to open an Assembler window lazarus bails with the attached lldb output.

The exceptions with my main app can be "continued" though. However when I close the app I get the attached error - don't know if its relevant or related to any of this, hence whyI include it.

Cheers

[Martin_fr]

As I expected.
Bit of background.

"DWARF" => This is the name of a debug info format. It is the format used on Mac. (And other OS)
It tells the debugger where variables (and other things) are in your exe.
So if you want to see "Form1" then the debugger looks at the dwarf info, and finds the address at wich Form1 is in memory. It also finds out what fields a "TForm" has.

Now when loading of this fails the debugger in the IDE does not even create instances of some classes. (It has no data to store in them).  Maybe that will change in future, but for now...

Some code did/does not check the "nil" value of those objects => crash.
So the crashes can be fixed easy. (And I will look at the asm window)

But the data is still not there. And we need to find out why.

I went through the logfile that you supplied, and compared it against the 2 crashes.
The dwarf is attempted to be loaded in "TDwarfLoaderThread.Execute;". This is the default, and it is confirmed by the log.

Actually the "debugln" I ask you to add are probably in the wrong place.  (But leave them in, they are not harmful either)

I just went through the date again, and I made some wrong conclusions the first time.

So here goes again. Lets start with
  TDwarfLoaderThread.Execute;
  line 290 onwards
  components\lazdebuggers\lazdebuggerfplldb\fplldbdebugger.pas

This is the unmodified 2.0.0 prior to applying the "try except"
So it does not have any other fixes branch modification, that you may not yet have.

You may also follow the svn link, and download the entire new file. (keep a backup of your old file).
The new file may compile, but I am not sure, it may need 1 or 2 other files.

In the below I have added some debugln.

Code: Pascal  [Select][+][-]

1. procedure TDwarfLoaderThread.Execute;
2. var
3.   AnImageLoader: TDbgImageLoader;
4. begin
5.   debugln(DBG_VERBOSE, ['THREAD TFpLldbDebugger.LoadDwarf ']);
6.   AnImageLoader := TDbgImageLoader.Create(FFileName);
7. debugln('THREAD AFTER TDbgImageLoader.Create'); ////////////////////////////////
8.   if not AnImageLoader.IsValid then begin
9.     debugln(['THREAD NOT VALID ']); ////////////////////////////////
10.     FreeAndNil(AnImageLoader);
11.     exit;
12.   end;
13.   debugln(['THREAD VALID ']); ////////////////////////////////
14.   if Terminated then
15.     exit;
16.  
17.   FImageLoaderList := TDbgImageLoaderList.Create(True);
18.   AnImageLoader.AddToLoaderList(FImageLoaderList);
19.   debugln(['THREAD LOADERLIST 1 ']); ////////////////////////////////
20.   if Terminated then
21.     exit;
22.   debugln(['THREAD LOADERLIST 2 ']); ////////////////////////////////
23.  
24. {$IFdef WithWinMemReader}
25.  
26.  

We probably need more changes.... Including some of the ones already in fixes branch.

Of course 2.0.2 is due in the next 2 weeks. So we can wait. On the otherhand, anything we get done by this Friday (maybe Saturday) may still get into 2.0.2.

As I said you can try to download entire files from the svn link. There are more details on which files to get:
https://forum.lazarus.freepascal.org/index.php/topic,42869.0.html

Or you can checkout from svn:

Code: [Select]

svn co https://svn.freepascal.org/svn/lazarus/fixes_2_0  target_folderTo update later: svn update

And then (IIRC) in that folder just run

Code: [Select]

make bigide LCL_PLATFORM=carbon CPU_TARGET=i386or

Code: [Select]

make bigide LCL_PLATFORM=cocoa CPU_TARGET=x86_64
You can do that in a separate folder. So it will not mess with your install.
In that folder create a file (with a text editor)
 lazarus.cfg
containing one line:
--primary-config-path=/Users/yourname/laz_2_conf

You can choose a diff path of course, and you need to create this folder. This file will make sure, that the 2nd Lazarus uses its own config. (You need to configure it from scratch)

This also allows you to follow the fixes branch.

---
One more alternative. Depending on how many patches/changes I would have to ask you to apply for testing, another option may be preferable.
Obviously those testing changes do not go to svn. But I do run my private git mirror, to which I can push them. (So you do not need to apply them by hand)
It is as (almost) simple as the svn above.

Code: [Select]

git clone https://github.com/User4martin/lazarus.git target_dir
cd target_dir
git checkout -f fixes_2_0
The -f to override local changes (if you later have some)
To update:  git pull
Or if you have local changes: git stash save; git pull; git stash pop

And then build as above and create the lazarus.cfg file

[ahnz]

Of course 2.0.2 is due in the next 2 weeks. So we can wait. On the otherhand, anything we get done by this Friday (maybe Saturday) may still get into 2.0.2.

Happy to help where I can

This is the option I went for ....

Or you can checkout from svn:
svn co https://svn.freepascal.org/svn/lazarus/fixes_2_0  target_folder
To update later: svn update

Incidentally, I think the link should be https://svn.freepascal.org/svn/lazarus/branches/fixes_2_0/

And then (IIRC) in that folder just run
make bigide LCL_PLATFORM=carbon CPU_TARGET=i386


You can do that in a separate folder. So it will not mess with your install.
In that folder create a file (with a text editor)
 lazarus.cfg
containing one line:
--primary-config-path=/Users/yourname/laz_2_conf

So I now have rev 60918 running in a seperate directory

My barebones app compiles and runs Ok with Debug, but I get the warning... The debugger encountered some errors/warnings ..... "Failed to load Dwarf debug info"

I can step through the app but still can not view any local variables. The attached lldb4.txt shows lldb output from the point where the app starts run and the breakpoint point has been stepped out of - I don't know what else I can give you to help figure out why local variables aren't viewable

Attempting to open an Assembler windows still bails. The attached lldb5.txt is the lldb output from the exception attempting to open an Assembler debug window

[ahnz]

Ok, so this weird.

I had a thought that maybe the fact the project was on my NAS might be responsible. I have had strange issues in the past where other, entirely unrelated things worked better locally than over an AFP share on a NAS. So I copied the project to my local machine.

And it worked.. ONCE!

I could step through and watch local variables. So I modified the code to include a call to a new function from a for loop. But then the debugger wouldn't even stop on the break point. So I took the for loop out.. No different. Then after a few attempts I also started to get the message "Oops the debugger entered the error state. Save your work now!"

I've tried copying the project locally again - no different

I've even rebooted - no different!!!!

Now every tie I try and debug, I get the "Oops" message

Is it possible that lldb save something somewhere which is getting corrupted??


UPDATE:

I'm looking to get some lldb output for this and I noticed a lot of these appearing....

Code: [Select]

The timestamp of the object-file "/Users/andy/Lazarus/lcl/units/i386-darwin/carbon/carbontabs.o" does not correspond to the timestamp (11/04/2019 08:51:15 am) stored inside the executable. The debug-info in this file is not loaded.
More to come

[Martin_fr]

I had a thought that maybe the fact the project was on my NAS might be responsible. I have had strange issues in the past where other, entirely unrelated things worked better locally than over an AFP share on a NAS. So I copied the project to my local machine.

And it worked.. ONCE!

I will look at why the IDE may not load the files.... and add debugln.

Now every tie I try and debug, I get the "Oops" message

Is it possible that lldb save something somewhere which is getting corrupted??


UPDATE:

I'm looking to get some lldb output for this and I noticed a lot of these appearing....

Code: [Select]

The timestamp of the object-file "/Users/andy/Lazarus/lcl/units/i386-darwin/carbon/carbontabs.o" does not correspond to the timestamp (11/04/2019 08:51:15 am) stored inside the executable. The debug-info in this file is not loaded.

So that would mean you can't step into the the LCL.

I am assuming that for you (in your timezone) this time was already in the past, at the time you got that msg? (for me it is yet to come)

You may want to try to do a "clean" build of your app.  (there should be an entry in the run menu).

[Martin_fr]

I attached a patch with some debugln.

If you could apply it

Code: [Select]

cd your_laz_fixes_dir
patch -p1 <debugln.diff
And then create a logfile with the error (no watches) happening.

Thanks

[ahnz]

I am assuming that for you (in your timezone) this time was already in the past, at the time you got that msg? (for me it is yet to come)

Yep, I'm in New Zealand and currently we're GMT+12

You may want to try to do a "clean" build of your app.  (there should be an entry in the run menu).

I did a Clean & Build and it init make any difference

I attached a patch with some debugln.

If you could apply it
cd your_laz_fixes_dir
patch -p1 <debugln.diff

Result:

patch -p1 <debugln.diff

patching file components/fpdebug/fpdbgdwarfdataclasses.pas
patching file components/fpdebug/fpdbgloader.pp
patching file components/fpdebug/fpimgreaderbase.pas
patching file components/fpdebug/fpimgreadermacho.pas
patching file components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas
Hunk #1 FAILED at 304.
Hunk #2 succeeded at 345 (offset 19 lines).
1 out of 2 hunks FAILED -- saving rejects to file components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas.rej

This is the first time I've applied patches like this, so I'm not sure what to do with this.

Reject file is attached along with the pre and post-patch versions of fplldbdebugger.pas

Regards

[Martin_fr]

Oh, I see what happened.

I did create the patch based on the fixes branch. But I had merged another revision to fixes already. And you had not run "svn update" yet.

There are several way forwards.

1) You can update

If you run  (cd laz_folder)
  svn update

it may either
- work fine
- report a conflict (since you have changes, after applying the patch), if ask answer "tf"  (their file / which will give up your local changes)

If it works fine => apply the patch again. It should tell you that some parts were already there, and do the remainder

Or if it works fine, or you get a conflict.
  svn -R revert  .
will revert all your changes

And then apply the patch again

2) You can leave the patch as it is.
The important bits have been applied, and it should compile.

If you look at the *.rej file, you can open components/lazdebuggers/lazdebuggerfplldb/fplldbdebugger.pas
go to line 300 and add the debugln yourself.

the indent changed, so the line differs on spacing, and patch did not like that. (there must be a way to tell it, to ignore that, but....)

[ahnz]

Ok, now I have rev. 60971 plus your patch.

I did get an error when recompiling the IDE after the patch at 210 of FpImgReaderBase.pas. Semi-colon missing ...

Code: Pascal  [Select][+][-]

1.     if (FileExists(s)) then AFileName := s
2. debugln(['TDbgFileLoader ',AFileName]);
3.  

Adding the semi, all compiled Ok.

Clean & Build, followed by Run (with Debug) of the barebones app produced the same "Oops" message, momentarily followed by the main form of the app being displayed

The breakpoint in the app click event shows

Invalid (On)
Hitcount: 0
Action: Break
Condition:

Code: Pascal  [Select][+][-]

1. procedure TForm1.Button1Click(Sender: TObject);
2. var
3.   s: string;
4.   i : integer;
5. begin
6.   s := 'That Tickles, ';            // <----- Breakpoint is here
7.   Label1.Caption := s;
8. end;

The debug log shows only

TCarbonButton.SetFocus Error: SetKeyboardFocus  failed with result -30585
TCarbonButton.SetFocus Error: SetKeyboardFocus  failed with result -30585
TCarbonButton.SetFocus Error: SetKeyboardFocus  failed with result -30585

An lldb output of the sequence of events is attached


Cheers

[Martin_fr]

Did you start the IDE with the entire

Code: [Select]

--debug-log=/path/to/yourfiles/laz.log --debug-enable=DBG_CMD_ECHO,DBG_STATE,DBG_DATA_MONITORS,DBGMI_QUEUE_DEBUG,FPDBG_DWARF_ERRORS,FPDBG_DWARF_WARNINGS,FPDBG_DWARF_VERBOSE_LOAD,FPDBG_DWARF_DATA_WARNINGS,DBG_VERBOSE,DBG_WARNINGS,DBG_STATE,DBG_EVENTS,DBG_THREAD_AND_FRAME?
Make sure DBG_VERBOSE is in the line.

Because some of the info is missing.

What I found from that log:

1)
Something else went wrong. lldb (the one used by the IDE, not the one you run on the outside) itself refused to start your app. Unfortunately (probably due to the commandline above not present?) the reason why lldb refused, was not captured.

2)
I believe this run did not have the issues that lead to the initial crash (even if that had not been fixed)
The class that would have had to be nil to cause the crash, seems to have been created (there is a log entry that was written very late in its constructor, very small change that it would still have crashed out after that)

3)
Many files were not loaded. (Some were).
I need to add some debuggln what is wrong with that timestamp

components/fpdebug/fpimgreadermacho.pas
line 184
There is a debugln, replace it with

Code: Pascal  [Select][+][-]

1.           debugln(Format('The timestamp of the object-file "%s" does not correspond to the timestamp (%s) stored inside the executable. The debug-info in this file is not loaded.', [DwarfDebugMap.ObjectFile, DateTimeToStr(FileDatetoDateTime(DwarfDebugMap.ObjFileAge)), ' <> ', DateTimeToStr(FileDatetoDateTime(FileAge(ASubFiles[i])))]))
2.  

The current debugln start the same, just something added at the end.

I suspect that somehow maybe the time is compared in diff timezones. But lets see.
And also need to find out, if fpc has the same issue.

Or there is an issue in fpc, and it writes the wrong time... That would be a problem...

[ahnz]

Did you start the IDE with the entire
--debug-log=/path/to/yourfiles/laz.log --debug-enable=DBG_CMD_ECHO,DBG_STATE,DBG_DATA_MONITORS,DBGMI_QUEUE_DEBUG,FPDBG_DWARF_ERRORS,FPDBG_DWARF_WARNINGS,FPDBG_DWARF_VERBOSE_LOAD,FPDBG_DWARF_DATA_WARNINGS,DBG_VERBOSE,DBG_WARNINGS,DBG_STATE,DBG_EVENTS,DBG_THREAD_AND_FRAME
?
Make sure DBG_VERBOSE is in the line.

Apologies - my bad

components/fpdebug/fpimgreadermacho.pas
line 184
There is a debugln, replace it with

Done!

This was run with

/Users/andy/Lazarus/lazarus.app/Contents/MacOS/lazarus --debug-log=/Users/andy/Desktop/debug_log.txt --debug-enable=DBG_CMD_ECHO,DBG_STATE,DBG_DATA_MONITORS,DBGMI_QUEUE_DEBUG,FPDBG_DWARF_ERRORS,FPDBG_DWARF_WARNINGS,FPDBG_DWARF_VERBOSE_LOAD,FPDBG_DWARF_DATA_WARNINGS,DBG_VERBOSE,DBG_WARNINGS,DBG_STATE,DBG_EVENTS,DBG_THREAD_AND_FRAME --primary-config-path=/Users/andy/laz_2_conf

I did a Clean & Build on the barebones app followed by a Run with debug

Local time is 12:15 on 15-April

Debug log is attached

Cheers

[Martin_fr]

Just for clarification:

It is your project that is on the NAS (except the compiled app?):
/Users/andy/Documents/BareBones/project1.app
/mnt/NAS/Documents/Andy/Pascal/src/BareBones/lib/i386-darwin/project1.o
/mnt/NAS/Documents/Andy/Pascal/src/BareBones/lib/i386-darwin/unit1.o

But not the IDE, neither the packages of the IDE:
/Users/andy/Lazarus/lcl/units/i386-darwin/carbon/carboncalendarview.o

[Martin_fr]

Short update:

1) The problem that caused the original crash (and probably also the inability to watch variables/locals), seems to have gone. At least it was again not present. Anyway the crash is fixed.

2) The reason for the failed launch (debug session aborted before project is started) is a filename matching the IDE trigger for another error message. Working on a fix.

3) The reason some debug info is not loaded (timestamp issues).
Still no good explanation. This is something that likely the linker would have caused. And the linker is part of Apple's tool chain (developer tools). Not much that can be done.
A clean build may help (including clean build of packages). Yet it seems mostly package files (lcl and co) are affected. So if you do not need to step into them, then this may not be an issue.
Also it is possible that this particular part will be better with "external debug info". Though I am not sure.