Recent

Author Topic: Debugging problem on Windows 11 ARM  (Read 6026 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #30 on: July 01, 2022, 09:03:09 pm »
Can't tell exactly if it is the same.

All your threads, almost all the time report
   IP=$00010002

If there really is code there, or if this is the OS idle location => then this would be entirely different.

Are you expecting your threads to be idle, or to be in some sort of "I am waiting" call?




EDIT: your log looks different :(

Also let me know on what you expect your threads to do.
Because if this 00010002 is some sort of placeholder => that will make it a lot harder.

By any chance, might that be in any external DLL?

If, so I have seen previously issues with 3rd party dll, that would react unkindly if the app in which they were loaded was showing signs of being debugged. Not saying this is the case, but must be considered.
« Last Edit: July 01, 2022, 09:11:53 pm by Martin_fr »

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #31 on: July 02, 2022, 10:29:15 am »
I tried to reproduce the problem with a simple project, adding a simple thread: No problem...
I then added another thread + TEvent in the TThread.Execute (which I use quite often):  Again no problem...

I will try to narrow down the problem by disabling all specific DLL needed by the project causing errors and to disable the threads one by one to find what's going on.

Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #32 on: July 02, 2022, 02:09:47 pm »
Just some background on what I am currently looking on. And why it does not match your case.


In my testcase I have 9 threads, that all go round and round and round on about 12 asm instructions.
So quite often more than one of them is at the same address, and at the same breakpoint.

If 2 threads hit a breakpoint, windows will inform the debugger twice, and each time it informs the debugger is stops all threads (at least it is supposed to, and does for me).
But in order to get the 2nd notification, the debugger must acknowledge the first. And at this time all threads will run again (well the debugger suspends some of them. But given enough threads and the right pattern some will run).

That also means that the following can happen.
(When there are more than 2 threads, then between the 2 breakpoint notifications, there can be other events. And the breakpoint may be removed in those other events. The below is massively simplified)

-- 2 threads hit breakpoint "A"
-> Notification for Thread-1 at "A"
    The debugger will handle the notification.
    But coincidently at this time, while handling the notification, the breakpoint "A" gets permanently removed.
<- Debugger acknowledges the notification, and app runs again (the other threads will be suspended, but that actually does not matter)
-> Notification for Thread-2 at "A"
    The breakpoint no longer exists, but the thread had hit it before.
    Here the debugger has a bug. => As it does not see a breakpoint, it does not handle it correctly.
<- Debugger acknowledges the notification, and app runs again     

And now the thread with the unhandled breakpoint will crash.



1) Why is it different from your case?
-> The breakpoint must get removed (well you did toggle it, so that includes removal)
-> A notification of type "breakpoint hit" must reach the debugger for that removed breakpoint
    That I have not seen in your log.
-> Also the thread would fail at the address of the breakpoint (actually + 1 byte).
    And your thread fails at this $100002 address. Which was never touched.
    That is, if that is indeed correctly reported....
    I can only guess it must be in a dll... Because your normal code resides at $4....... addresses. And the Kernel usually is at $7f.......



2) What exactly means "does not handle the breakpoint correctly"

Well here is how a breakpoint works (though of course WOW64 for arch/arm will (should) emulate that).

Say the testcase has an instruction like  (this is just dummy data)

Code: Text  [Select][+][-]
  1. Address     mem-dump       asm
  2. 0x410010  71 72 73 74     mov rax, eax

The debugger will replace the first byte with $CC (an int3 instruction, that is handled as breakpoint).

The test process will execute the int3, and its instruction pointer will then be at 0x410011. The byte after the breakpoint.
The OS catches the int3 and tells the debugger.

Now the debugger must do 2 things:
- tell the OS, to clear the signal. So the test app does not get a signal for the int3. (that seems still to happen in the buggy case)
- restore the original code, and restore the instruction pointer to 0x410010 => so the original instruction can be run
- (after that, put the breakpoint back in place)

But due to the bug, the debugger does not do the 2nd step.

That means the test app continues at 0x410011. And it takes the "72 ..." as it's next instruction. Only that was never an instruction, and that is why it crashes.


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #33 on: July 04, 2022, 06:05:46 pm »
Ok, I made some changes to git main. Fixing the issue I mentioned.

I have no idea if it affects your issue. You can test it https://gitlab.com/martin_frb/lazarus/-/commits/f-fpdebug-new-win-brkpoint

You will need to build a 2.3 Lazarus for this. The link is based on the current main branch.

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #34 on: July 05, 2022, 03:44:03 pm »
Thanks Martin,
I didn't have time yet to find the thread or dll problem. I will let you know ASAP.
Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #35 on: July 06, 2022, 11:56:31 am »
For information,

I got an error while building FPC 3.2.2 with FPCUPdeluxe in file gdbint.pp

Code: Pascal  [Select][+][-]
  1. Start compiling package gdbint for target x86_64-win64.
  2. File libgdb.a not found
  3.        Compiling gdbint\src\gdbint.pp
  4. The installer encountered the following error:
  5. External command "C:/fpcupdeluxe/fpcsrc/compiler/ppcx64.exe -Twin64 -FUgdbint\units\x86_64-win64\ -FuC:\fpcupdeluxe\fpcsrc\rtl\units\x86_64-win64\ -Figdbint\src -Fl -Ur -Xs -O2 -n -vw-n-h-l-d-u-t-p-c- -dFPC_SOFT_FPUX80 -dx86_64 -dRELEASE -Sc -viq gdbint\src\gdbint.pp" failed with exit code 1. Console output:
  6. Target OS: Win64 for x64
  7. Compiling gdbint\src\gdbint.pp
  8. User defined: using gdb 6.6.x
  9. gdbint.pp(3726) Error: Invalid DLL C:\WINDOWS\system32\kernel32.dll, invalid header size
  10. gdbint.pp(3726) Error: Invalid DLL C:\WINDOWS\system32\user32.dll, invalid header size
  11. Fatal: There were 2 errors compiling module, stopping
  12. Fatal: Compilation aborted
  13. make.exe[1]: *** [packages_all] Error 2
  14. C:\fpcupdeluxe\fpcbootstrap\make.exe: *** [build-stamp.x86_64-win64] Error 2
  15.  
  16. fpcupdeluxe: ERROR: FPCNativeInstaller (BuildModuleCustom: FPC): Error running C:\fpcupdeluxe\fpcbootstrap\make.exe for FPC failed with exit code 2
  17. . Details:
  18.  
  19.  
  20. ERROR: Fpcupdeluxe fatal error !
  21. Sequencer (FPC): Failure running fpcupdeluxe: error executing sequence FPC
  22. Sequencer (Default): Failure running fpcupdeluxe: error executing sequence Default
  23.  


I had to comment out the 2 lines in gdbint.pp (701 and 702) and check "FCP/Laz Build Only in "Setup +", and build only FPC.

Then the Lazarus Update went well.

Is there a way to copy/apply the settings from one Lazarus install to another? I mean having all user settings and needed packages installation done quickly?


Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #36 on: July 06, 2022, 02:50:59 pm »
Is there a way to copy/apply the settings from one Lazarus install to another? I mean having all user settings and needed packages installation done quickly?


Not 100%

Your settings are in a collection of xml files in the primary config path (menu: view > ide internals > about ide => then search for it)
Normally  C:\Users\USERNAME\AppData\Local\lazarus

editoroptions.xml can afaik be copiend. (has no paths)
environmentoptions.xml contains paths. And those must be adapted.

Best to check all file for paths.
If it is a regular think, make sure your paths all are using macros like ($lazarusdir) / not sure if that is correct....


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #37 on: July 06, 2022, 02:52:31 pm »
I got an error while building FPC 3.2.2 with FPCUPdeluxe in file gdbint.pp
Maybe report this in the fpcupdeluxe thread ?
https://forum.lazarus.freepascal.org/index.php/topic,34645.0.html

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #38 on: July 07, 2022, 02:20:31 pm »
Quote
Maybe report this in the fpcupdeluxe thread ?
Done

Finally, I was able to reproduce the problem with a simple project (Lazarus 2.3.0 and FPC 3.2.2 + Cross Compile winx64->winx32 ). It is related to TThread and TEvent.
The reported address is still the same on my computer:

Here is the sample project and log.

To reproduce the problem you must have at least 2 "TThreadEvent" running:

    - Start the program
    - Click on Start Thread 1
    - F9 when breakpoint is hit on line 192  "StartThread(MyThread1, TButton(Sender), 'MyThread1');"
    - Click on Start Thread 2
    - Click on Start Thread 3
    - Then try to add and remove (several time) a breakpoint at line 197 "StartThread(MyThread2, TButton(Sender), 'MyThread2');"

You should have the error message box (repeating when you click the IDE run green arrow).


Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #39 on: July 12, 2022, 08:12:47 pm »
Hi,

Am I the only one having this problem?
Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #40 on: July 12, 2022, 09:20:52 pm »
I haven't found time yet to look at your example, and see if I could reproduce....

It may be a while...

Luc

  • Jr. Member
  • **
  • Posts: 59
Re: Debugging problem on Windows 11 ARM
« Reply #41 on: July 12, 2022, 10:12:39 pm »
No problem :)
Win 11 ARM (Running in Parallel VM - Mac OSX) - Lazarus 2.2.6 - i386-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9754
  • Debugger - SynEdit - and more
    • wiki
Re: Debugging problem on Windows 11 ARM
« Reply #42 on: April 18, 2023, 03:02:02 pm »
I had another look at this.

I still can't reproduce it. But I don't have an arm machine. So the closest I have is Win-11 inside a VM (Virtualbox) running on a 64bit Intel CPU.
I did cross-compile your example (using Lazarus 2.2.4 with 3.2.2).
But nothing....

I looked at the log you provided.

There are several
Code: Text  [Select][+][-]
  1.     Failed to suspend Thread 6356 (handle: 2072). Error: 87: The parameter is incorrect.
which I don't get.
From the looks at it, I believe those are threads started by dll loaded during your app starts. Those dll may be from Windows, but they could be from 3rd party products (afaik, some dll are installed in ways, such that all apps will load them).
This may or may not be related, but as it is there is no indication that this has anything to do with the later error.
It may be a user-rights issue... I.e., if a dll (like an antivirus) started a thread, then it may have elevated it, and protected it against interference from a debugger.


The set/unset of the breakpoint looks all as it should. All fine.

And then the error happens
Code: Text  [Select][+][-]
  1.   EVENT for Process 14368 Thread 7060: EXCEPTION_DEBUG_EVENT Code:C000001D Flags:0 NumParam:0 EXCEPTION_ILLEGAL_INSTRUCTION True
  2.   Process stopped with event deException. IP=$00010002, SP=$0F5DFD38, BSP=$0F5DFD50. HasBreak: False
  3.  

The only issue is that the crash happens at $00010002 wich is either a special address or in the kernel. And I don't see how the debugger could have influenced it.

At this point I would consider that it could also be an issue either in Windows 11 itself (maybe in arm related code, or emulating intel). Or an issue with the VM.
It could be timing related / race condition. And thus by chance not be seen with gdb.





 

TinyPortal © 2005-2018