Recent

Author Topic: Cannot 'STOP' An Application in Debugger  (Read 2405 times)

kevin.black

  • Full Member
  • ***
  • Posts: 122
Cannot 'STOP' An Application in Debugger
« on: March 06, 2019, 12:52:05 am »
Using 2.0.0RC3 on Mojave.

When I try to stop an application that I'm running with the debugger, pressing the RED stop button does nothing. When I try to quit lazarus, I get asked if I want to stop the debugger, saying I do does nothing. The only way I can stop the debugger is to go to the task bar and Quit lazarus or force quit lazarus.

Now I may have the debugger setup incorrectly, but I have generally accepted the defaults:

Dwarf with sets(-gw -godwarfsets)
LLDB Debugger (with fpdebug)(Beta)
Optimisation Level 1 (Debugger Friendly)

So see image, if I even try to quit lazarus, I'm asked to stop the debugger, I do that and try to quit, but the message appears again.

If I look in the Activity Monitor (other than Lazarus) I can see a process corresponding to the app being debugged)?

Any ideas where I'm going wrong?

Kevin

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5389
    • wiki
Re: Cannot 'STOP' An Application in Debugger
« Reply #1 on: March 06, 2019, 01:07:34 am »
Is that always, for all apps? Even an empty project, with an empty Form?

If not, is your app a GUI app or console?
Also 32bit or 64 bit.

Using 2.0.0RC3 on Mojave.
While nothing is known between RC3 and the release that matches your description, could you still try the 2.0 release? (Can be deferred if not practical right now)

Quote
When I try to stop an application that I'm running with the debugger, pressing the RED stop button does nothing. When I try to quit lazarus, I get asked if I want to stop the debugger, saying I do does nothing. The only way I can stop the debugger is to go to the task bar and Quit lazarus or force quit lazarus.
Please try: "reset debugger" from the run menu.

If the issue only happens on certain, but not all debug sessions, then you may want to open the Options (Menu Tools), and under Debugger/General enable "Always reset debugger"

Quote
Now I may have the debugger setup incorrectly, but I have generally accepted the defaults:

Dwarf with sets(-gw -godwarfsets)
LLDB Debugger (with fpdebug)(Beta)
Optimisation Level 1 (Debugger Friendly)
And I assume in the options the edit field below "LLdb Debugger (with fpdebug)..." is set to: /usr/bin/lldb (or where ever your lldb is located)

Quote
If I look in the Activity Monitor (other than Lazarus) I can see a process corresponding to the app being debugged)?
You can shot the process with "kill" or "kill -9" and the pid of the process. But that is not a solution.


If nothing else, then please provide a logfile as specified under:
http://forum.lazarus-ide.org/index.php/topic,42869.0.html
« Last Edit: March 06, 2019, 01:09:44 am by Martin_fr »

kevin.black

  • Full Member
  • ***
  • Posts: 122
Re: Cannot 'STOP' An Application in Debugger
« Reply #2 on: March 07, 2019, 12:02:11 am »
@Martin,
Is that always, for all apps? Even an empty project, with an empty Form?
No, The app in question is a GUI app and a (statically loaded) DYLIB. I have a TEST app that is a simpler application with a simpler statically loaded DYLIB (it does the same thing, but way less complicated). I can stop the debug session with the 'simple app. BUT BUT BUT in both cases, if I have a breakpoint set and RUN the application, ie. it behaves as previously described):
  • The IDE does not hide/disappear as configureds
  • The initial 'show message' does not happen (ie. simply to let me know that the code is in the initialization section of the GUI app)
  • No matter what I do I cannot close the debug session as described
-
If not, is your app a GUI app or console?
Also 32bit or 64 bit.
GUI/64bit/cocoa
While nothing is known between RC3 and the release that matches your description, could you still try the 2.0 release? (Can be deferred if not practical right now)
Probably impractical, but I have fpupdeluxe installed with both FIXES and TRUNK. I can try in there?
Please try: "reset debugger" from the run menu.
Works in the smaller/simpler test app, not in the other/main app, works in neither if a breakpoint is set.
If the issue only happens on certain, but not all debug sessions, then you may want to open the Options (Menu Tools), and under Debugger/General enable "Always reset debugger".
Always in one of the apps (the main and more complicated app), but only with a breakpoint set in the smaller test app.
And I assume in the options the edit field below "LLdb Debugger (with fpdebug)..." is set to: /usr/bin/lldb (or where ever your lldb is located).
Correct.
Quote
If I look in the Activity Monitor (other than Lazarus) I can NOT see a process corresponding to the app being debugged)?
And my comment was missing a word (NOT), I have amended my comment above. When the app starts correctly I see the App name in the Activity Monitor and I can terminate it. BUT when I have the debugger issue (and the code doesn't even get to the initialisation section of the application it is as if the app has not started, yet the debug session has started and hangs). I look in the Activity Monitor and I cannot see the application (which is why I have to quit Lazarus to get the debug session to terminate).
You can shot the process with "kill" or "kill -9" and the pid of the process. But that is not a solution.
See above, I would if I could find a PID.
If nothing else, then please provide a logfile as specified under:
http://forum.lazarus-ide.org/index.php/topic,42869.0.html
I'll see what I can do. Thanks for your help Martin.

Kevin
« Last Edit: March 07, 2019, 12:03:47 am by kevin.black »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5389
    • wiki
Re: Cannot 'STOP' An Application in Debugger
« Reply #3 on: March 07, 2019, 12:13:15 am »
works in neither if a breakpoint is set.

Quote
If I look in the Activity Monitor (other than Lazarus) I can NOT see a process corresponding to the app being debugged)?
And my comment was missing a word (NOT), I have amended my comment above. When the app starts correctly I see the App name in the Activity Monitor and I can terminate it. BUT when I have the debugger issue (and the code doesn't even get to the initialisation section of the application it is as if the app has not started, yet the debug session has started and hangs). I look in the Activity Monitor and I cannot see the application (which is why I have to quit Lazarus to get the debug session to terminate).

Ok, so it is likely that when setting the breakpoint lldb returns some prompt that the IDE does not expect.

The IDE (currently) has no timeout. It waits forever for such a response. That is why you can not "stop" the debugger.

Once I have the log, I will see if I can figure out, how this affects "reset debugger" (from the run menu). Because this is supposed to not wait for anything. => Try to tell lldb to shutdown, but kill the lldb process in any case. Well supposed to....

kevin.black

  • Full Member
  • ***
  • Posts: 122
Re: Cannot 'STOP' An Application in Debugger
« Reply #4 on: March 07, 2019, 01:00:59 am »
@Martin,

Thanks, for the record and repeatable, the IDE is still visible, the debugger is active, pressing the RED button does nothing, Quitting lazarus jest gets the message 'Do you want to terminate the debugger' which doesn't terminate and the activity monitor does not show the application.

FWIW, the same issue with using the RED button, but in this case RESET DEBUGGER worked??? I have no idea why because it did not/does not work if I run lazarus from the shortcut.

And now IF I put in a breakpoint, the app runs and the DEBUGGER enters an error state, This again does not happen when I run from the shortcut.

EDIT: I have rerun from the shortcut (not the terminal). And although the app does not appear etc etc, it now looks like RESET DEBUGGER is working. The ONLY change I've done (and I cannot see how this would be an issue) is to take the spaces out of the folder name (was 'EMPSecure REST API' is now 'EMPSecureRESTAPI')?

EDIT EDITAnd now I run with debugger, the application starts, it gets an error (BAD_ADDR) and I click the RED button and am told execution stopped, and the RED button worked. Boy, is this confusing. What was 100% repeatable failure, now works, will try with a breakpoint included.

EDIT EDIT EDITOK, so restarted lazarus and the problem was the same, except that I could reset the debugger. This happened with about 5 attempts to run the application with the debugger. On the 6th attempt it actually worked. The IDE was hidden and the application ran. It terminated after pressing the RED button. After then another 4 failed attempts to start the application, the fifth attempt worked AND the debugger stopped as it should, at a breakpoint. It terminated by pressing the RED button. Could not get the debugger to run so rebuilt the project, after two attempts started to run, stopped at the breakpoint and hung when I stepped to the next line. Application had the rainbow wheel of death. Reset the debugger and the application continued running from where it had stopped with the debugger. And then after yet another rebuild and many attempts to run finally got it to run, and immediately said the debugger had failed, so stopped it and the program continued on without the debugger. The absolute lack of consistency (both in running correctly and in running incorrectly) is very frustrating.

Logfile attached.

Thanks,
Kevin
« Last Edit: March 07, 2019, 04:12:44 am by kevin.black »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5389
    • wiki
Re: Cannot 'STOP' An Application in Debugger
« Reply #5 on: March 07, 2019, 03:01:09 pm »
Ok, I found some useful info in the log. (Though nothing on why "reset" would not work.)

First of all, do you currently have debug info set to "external" ? If not then you did in the past?
The debugger complains that the files in (*.dSYM the folder for external debug info)
/Users/kevin/Dropbox/Lazarus/EMPSecureRESTAPI/x86_64-darwin/EMPServerAPITest.dSYM
is outdated.
So this part of the debug info is ignored (which may mean that some symbols will not show, or some breakpoints will not break)

The debugger then resorts to reading the *.o files.
But they are outdated too:
   "error: libdbxserverapi.dylib debug map object file '/Users/kevin/Dropbox/Lazarus/EMPSecureRESTAPI/lib/x86_64-darwin/synacode.o' has changed (actual time is 2019-03-07 10:39:09.000000000, debug map time is 2019-03-07 10:30:37.000000000) since this executable was linked, file will be ignored"

I do not know why this files are outdated, usually this happens, if you compile some units again, but do not build (link) the exe.

I believe you have a main app, and a dll? Maybe both use the same files, so compiling one is out-dating the other?

Maybe dropbox changes the timestamps?
But some info (in dSYM) is detected as wrong, due to internal unique identifiers. So that is definitely outdated.



Btw app + lib:
This has actually not been tested. The library may have its own exception handlers, and I do not know if exceptions will be caught in both, the app and the lib (there are restrictions with gdb too).



Back to the issue.

While starting the app, lldb gives warnings about those outdate files. Unfortunately they are prefixed "error". So the IDE believes lldb encountered a (fatal) error. The IDE sets the (Lazarus part of the) debugger to error-state.
But lldb still launches the app.

The IDE does not accept stop, since error-state means it believes the app is already stopped.

As I said "reset" (from run menu) should work. Because it does not check the state.


"Always reset debugger" from the options, is not the same. This will only reset the debugger before the next debug session.
So it will leave the debugger in whatever state it is, until you try to start a new debug session.

It also makes no difference here, because once error-state is set, a reset will happen before the next debug session.





In any case, you need to find why your debug info is screwed.

The IDE should definitely tell the user about the problem. (outdated debug info) And give the user a choice.

If the user ignores the broken debug info:
The IDE should not enter error state in case of those warnings. lldb still works

If the IDE enters error state, it should make sure the app is stopped. Otherwise this can be very confusing.

« Last Edit: March 07, 2019, 03:04:22 pm by Martin_fr »

kevin.black

  • Full Member
  • ***
  • Posts: 122
Re: Cannot 'STOP' An Application in Debugger
« Reply #6 on: March 07, 2019, 11:40:06 pm »
@Marti,

Wow, thank you for taking the time to forensically audit that log file. Your response is well and truly above anything I expected. If you were getting paid for this you could retire early :)
Ok, I found some useful info in the log. (Though nothing on why "reset" would not work.)
As noted in my EDIT updates, RESET the Debugger now appears to work fairly consistently. I cannot explain why, it was not working consistently previously and I don't think I changed anything?
First of all, do you currently have debug info set to "external" ? If not then you did in the past?
The debugger complains that the files in (*.dSYM the folder for external debug info)
/Users/kevin/Dropbox/Lazarus/EMPSecureRESTAPI/x86_64-darwin/EMPServerAPITest.dSYM is outdated.
So this part of the debug info is ignored (which may mean that some symbols will not show, or some breakpoints will not break)
Since I don't know enough to even know what that is, it is unlikely I would have purposefully set debug info to external. I don't even know (offhand) where that setting is. OK found it, it's possible I had that checkbox ticked at some time (I 'might' have inadvertently tried it). Regardless in the main app and the DYLIB it isn't checked - confirmed.
The debugger then resorts to reading the *.o files.
But they are outdated too:
   "error: libdbxserverapi.dylib debug map object file '/Users/kevin/Dropbox/Lazarus/EMPSecureRESTAPI/lib/x86_64-darwin/synacode.o' has changed (actual time is 2019-03-07 10:39:09.000000000, debug map time is 2019-03-07 10:30:37.000000000) since this executable was linked, file will be ignored"

I do not know why this files are outdated, usually this happens, if you compile some units again, but do not build (link) the exe.

I believe you have a main app, and a dll? Maybe both use the same files, so compiling one is out-dating the other?

Maybe dropbox changes the timestamps?
But some info (in dSYM) is detected as wrong, due to internal unique identifiers. So that is definitely outdated.
Yes I have an application that calls a DYLIB. I have them in s project group as they were in Delphi. As a rule I build the DYLIB then test with the test application usually (but I cannot guarantee) after building the Test App. I can see that for the debugger to work I MUST build the test app if I change the DYLIB otherwise it will essentially be linked to the 'old' version of the DYLIB and the debugger will detect and not be best pleased. And I also agree about Dropbox timestamping although since I am doing the changes here (and not elsewhere) the timestamps should be correct, so it is likely not rebuilding the Test App.
Btw app + lib:
This has actually not been tested. The library may have its own exception handlers, and I do not know if exceptions will be caught in both, the app and the lib (there are restrictions with gdb too).
I was simply following the Delphi 'rules'. If you start a DLL in Delphi in debug mode using the run parameters (ie. with a host application) you can debug the DLL, ie. set breakpoints etc etc

If that's not available in Lazarus (and I tried and it didn't seem to work), then can you suggest how I might debug a DLL/DYLIB other than just putting in a gazillion WriteLog/ShowMessage statements?
While starting the app, lldb gives warnings about those outdate files. Unfortunately they are prefixed "error". So the IDE believes lldb encountered a (fatal) error. The IDE sets the (Lazarus part of the) debugger to error-state.
But lldb still launches the app.

The IDE does not accept stop, since error-state means it believes the app is already stopped.
Well that little mystery is now explained, great thanks. It would be confusing if the App was running and the stop button was RESET. I think disabling the button (if RED) is probably not good. At least if the app isn't running pressing stop should do nothing other than change the colour of the button - just thinking out loud....
As I said "reset" (from run menu) should work. Because it does not check the state.
As noted above, it is working now even though I have no idea why.
"Always reset debugger" from the options, is not the same. This will only reset the debugger before the next debug session.
So it will leave the debugger in whatever state it is, until you try to start a new debug session.

It also makes no difference here, because once error-state is set, a reset will happen before the next debug session.
That makes sense.
In any case, you need to find why your debug info is screwed.
I think there are potentially three issues all of which you have identified:
  • The external debug info
  • The outdated DYLIB info, ie. I need to Build the Test App every time to make sure it links with the latest and available DYLIB
  • Dropbox timestamping, I have the source in Dropbox because If I rewind a VM to a previous snapshot or if the VM becomes corrupt, I can rebuild/restore the VM and the actual code and associated data will always be up to date, it has saved my backside several times (In my Delphi days, I also had the codebase in a SVN repository also in Dropbox - I haven't managed to set that up yet with Lazarus
The IDE should definitely tell the user about the problem. (outdated debug info) And give the user a choice.

If the user ignores the broken debug info:
The IDE should not enter error state in case of those warnings. lldb still works

If the IDE enters error state, it should make sure the app is stopped. Otherwise this can be very confusing.
Agreed with all, especially the bit about 'very confusing'

As noted in my update to the previous post, There have been instances where the Debugger worked as should, even to the point of single stepping through the code, instances where the debugger failed and the app continued blissfully on and instances where nothing happened, ie. the app didn't appear to load, but the debugger was sitting there with the red button not terminating just hanging.

Your response is excellent and much appreciated. Thanks you agin Martin for taking the time to help me. I now know what to do and, if I can workout how to debug a DYLIB correctly, I will indeed be home and hosed.

Regards,
Kevin

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5389
    • wiki
Re: Cannot 'STOP' An Application in Debugger
« Reply #7 on: March 08, 2019, 12:05:37 am »
... external debug info
Quote
it's possible I had that checkbox ticked at some time

I am not sure if the dSYM folder gets removed when you uncheck it.
In any case whenever you stop using it, make sure that you delete the *.dSYM folder.

Quote
I was simply following the Delphi 'rules'. If you start a DLL in Delphi in debug mode using the run parameters (ie. with a host application) you can debug the DLL, ie. set breakpoints etc etc
"Run params" is the way.

But there is a flaw, in our debugger(s).
To get exception catching working, a breakpoint is needed on fpc_raiseexception.

The IDE tells lldb (and gdb) to set this. This is done right after launching the app. So if you load the dll later (with openlibrary, or whatever it is called), then the library does not have that breakpoint.

If the dll is statically linked, and loaded by the OS, it may already be loaded... Or not. I did tests with gdb, and gdb can keep the request, but it depends on the correct mix of debug info between launcher app and library.
I did not do tests with lldb. But it is possible lldb is handling this smart enough.

You will know, if the debugger tells you when your app or library is raising an exception.

Btw, same goes for breakpoints that are set before you launch the debugger. If they work => good.

That is to say, you are the first to test the lldb based debugger with libraries.

Quote
If that's not available in Lazarus (and I tried and it didn't seem to work), then can you suggest how I might debug a DLL/DYLIB other than just putting in a gazillion WriteLog/ShowMessage statements?
As for normal breakpoints. If they do not work.

Start the app with only breakpoints in your app, but not in the library.
Have one breakpoint that is hit in your app, after the library was loaded. Once this is reached, you should be able to set breakpoints in the dll.

I know this is not very useful, if you need a lot of breakpoints...
It may be that disabling them at launch-time, and enabling them at the helper-breakpoint does the trick.

I havent tested any of this.

Most of the work so far was to get the debugger to work at all (for the main app).

I'll see if i can dig a bit deeper, but that may take some time.


kevin.black

  • Full Member
  • ***
  • Posts: 122
Re: Cannot 'STOP' An Application in Debugger
« Reply #8 on: March 08, 2019, 03:15:23 am »
Most of the work so far was to get the debugger to work at all (for the main app).
Look, I appreciate you doing this, I'm getting 'better' results.

That wasn't meant to say, that it shouldn't be improved. Just saying some patience will be needed.

And your feedback is hugely appreciated. Without such feedback, many issues would never be discovered.
« Last Edit: March 08, 2019, 03:29:25 am by Martin_fr »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5389
    • wiki
Re: Cannot 'STOP' An Application in Debugger
« Reply #9 on: March 10, 2019, 07:09:42 pm »
The IDE should definitely tell the user about the problem. (outdated debug info) And give the user a choice.

If the user ignores the broken debug info:
The IDE should not enter error state in case of those warnings. lldb still works

If the IDE enters error state, it should make sure the app is stopped. Otherwise this can be very confusing.

I updated trunk

In case of an error, the debugger should now kill the target app.

Also during startup, the debug-info errors should no longer give an error. A dialog should prompt what to do:
At that point the app will already be running, so choices are to continue (ignore) or to stop it.

The global options have a setting to disable the prompt (keep app running).

Note that once you did generate external debug info it will be in your project folder until you delete it (the IDE unfortunately does not do that).
That is also true for "new projects" (not yet saved), that are compiled in a temp directory. Once you got external info there, you get warnings.

kevin.black

  • Full Member
  • ***
  • Posts: 122
Re: Cannot 'STOP' An Application in Debugger
« Reply #10 on: March 19, 2019, 01:27:01 am »
Sorry for not getting back, distracted with other things.

I updated trunk

Thanks you for doing that, I appreciate you following it up.

Regards,
Kevin