Recent

Author Topic: Remote debugging - no assembler code displayed and no code execution pointer  (Read 12200 times)

ccrause

  • Hero Member
  • *****
  • Posts: 845
When debugging a remote AVR simulation running on simavr I don't see the assembler code in the assembler debug view.  I also don't see the location of the current line of code to be debugged.  See screen shot of a debugging session attached.  I can view the values of variables in the watch list and see the values update as I step through the code, so the basic debugger session works.

Anyone with ideas where I can start digging?

Ñuño_Martínez

  • Hero Member
  • *****
  • Posts: 1186
    • Burdjia
I didn't do such debugging, but since Lazarus is compatible with GDB may be you can use such debugger. If I remember correctly it has a quite stable remote debugging system.
Are you interested in game programming? Join the Pascal Game Development community!
Also visit the Game Development Portal

ccrause

  • Hero Member
  • *****
  • Posts: 845
I didn't do such debugging, but since Lazarus is compatible with GDB may be you can use such debugger. If I remember correctly it has a quite stable remote debugging system.
Some more detail (I realize I was a bit cryptic in the first post):
I'm using avr-gdb to debug a remote session with simavr as simulator to run the AVR code.  Lazarus is configured to connect to a gdbserver connection.  Lazarus stops at break points and display simple data types and register contents correctly. However Lazarus doesn't show the current execution point in the source file and also doesn't show code in the Assembler view.

My suspicion is that the GDB interface in Lazarus doesn't take into account the architecture difference between the debug target (embedded avr) and the host (linux x86-64), but I don't know how to check that.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9792
  • Debugger - SynEdit - and more
    • wiki
I will try to add more later / travelling now.

check the communication between IDE and gdb / window "debug output" or better via define when compiling the IDE.

http://forum.lazarus.freepascal.org/index.php/topic,37353.msg250966.html#msg250966

GDB Should report the location when stopping. (reach breakpoint).

IIRC in theIDE the function ProcessStopped.

there is a wiki page about this, and how to provide a logfile with all the gdb output.

ccrause

  • Hero Member
  • *****
  • Posts: 845
I've tested debugging for both AVR (via simavr for gdb server) and Linux using the same test project code:
Code: Pascal  [Select][+][-]
  1. program project1_linux_x86_64;
  2.  
  3. var
  4.   a, b, c: word;
  5.  
  6. function test(const a1, a2: word): word;
  7. begin
  8.   if a > a2 then
  9.     result := a1
  10.   else
  11.     result := a2;
  12. end;
  13.  
  14. begin
  15.   a := 250;
  16.   b := 5;
  17.   c := test(a, b);
  18. end.

This project was then compiled either for Linux x86_64 (development environment, FPC 3.0.2) and debugged with gdb or compiled for AVR (FPC 3.1.1 cross compiler) and debugged with avr-gdb. A breakpoint was inserted at line 17, the project was run up to the breakpoint and the debug output was saved as project1_linux_x86_64-debug.txt or project1_avr-debug.txt respectively.

The native Linux test obviously debugged correctly, with disassembled code and watches populated and the code execution line highlighted.  The AVR test doesn't show disassembled code, the watch (variable a) shows an incorrect value and no highlighting of the current execution point in the source code.

The debug output for the native Linux project shows that the break point has been reached, then calls -data-disassemble a couple of times:
Code: [Select]
=breakpoint-modified,bkpt={number="8",type="breakpoint",disp="keep",enabled="y",addr="0x000000000040021b",func="main",file="project1_linux_x86_64.pp",fullname="/home/christo/fpc/fpc-avr/src/examples/test/project1_linux_x86_64.pp",line="17",thread-groups=["i1"],times="1",original-location="/home/christo/fpc/fpc-avr/src/examples/test/project1_linux_x86_64.pp:17"}
~"\n"
~"Breakpoint 8, main () at project1_linux_x86_64.pp:17\n"
~"17\t  c := test(a, b);\n"
*stopped,reason="breakpoint-hit",disp="keep",bkptno="8",frame={addr="0x000000000040021b",func="main",args=[],file="project1_linux_x86_64.pp",fullname="/home/christo/fpc/fpc-avr/src/examples/test/project1_linux_x86_64.pp",line="17"},thread-id="1",stopped-threads="all",core="0"
(gdb)
<-data-disassemble -s 4194736 -e 4194737 -- 0>

Similarly the AVR project debug output shows that it reached the breakpoint, but then no disassemble calls, only calls to determine the type and value of variable "a" which is in the watch list:
Code: [Select]
=breakpoint-modified,bkpt={number="6",type="breakpoint",disp="keep",enabled="y",addr="0x0000011a",func="main",file="project1_avr.pp",fullname="/home/christo/fpc/fpc-avr/src/examples/test/project1_avr.pp",line="17",thread-groups=["i1"],times="1",original-location="/home/christo/fpc/fpc-avr/src/examples/test/project1_avr.pp:17"}
~"\nBreakpoint "
~"6, main () at project1_avr.pp:17\n"
~"17\t  c := test(a, b);\n"
*stopped,reason="breakpoint-hit",disp="keep",bkptno="6",frame={addr="0x0000011a",func="main",args=[],file="project1_avr.pp",fullname="/home/christo/fpc/fpc-avr/src/examples/test/project1_avr.pp",line="17"},thread-id="1",stopped-threads="all"
&"ptype A\n"
~"type = WORD\n"
^done
(gdb)
<whatis A>
&"whatis A\n"
~"type = WORD\n"
^done
(gdb)
<-data-evaluate-expression A>
^done,value="12090"
(gdb)
<-thread-info>

To me it seems as if the debugger communication with avr-gdb works fine, but for some reason the gdbmiserverdebugger doesn't request the same amount of information as gdbmidebugger. Attached please find detailed debugger logs and Lazarus IDE output logs for th two test cases.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9792
  • Debugger - SynEdit - and more
    • wiki
Is it correct for avr that the exe is loaded at very low addressed? 0x011a ?

----------------

* Ensure you compile with -O- (optimization 0 / none).
Even -O1 can screw up.

* You may want to try to switch between dwarf(2) and stabs, if there are issues, this may help (project options/debug)

* If avail, older gdb.  (on some platforms (win) gdb got worse above version 7.7)

----------------

My expertise is only about the IDE integration (communication to gdb).
* If the error is broken debug info by fpc or the linker, then that needs to be discussed with someone from the fpc team (fpc mail list). So it will be hard to tell if it is fpc or gdb.
* If the error is in gdb, then all you can try is a diff version of gdb

In both cases stabs vs dwarf may help.

----------------

As for the watch:
Code: Pascal  [Select][+][-]
  1. <-data-evaluate-expression A>
  2. ^done,value="12090"
  3. (gdb)
If gdb is returning the wrong value, then there is nothing the IDE can do.
Either gdb is at fault, or fpc generated bad debug info, or the linker messed it up.


--------
Disassembler:
How or why this affects the disassembler is not clear.
Please try (avr only)
http://wiki.lazarus.freepascal.org/GDB_Debugger_Tips#Log_info_for_debug_session

This includes the state of the IDE, so it gives some more info.

Change the --debug-enable option to:
 --debug-enable=DBG_CMD_ECHO,DBG_STATE,DBG_DATA_MONITORS,DBGMI_QUEUE_DEBUG,DBGMI_TYPE_INFO,DBG_DISASSEMBLER,DBG_LOCATION_INFO,DBG_BREAKPOINT,DBGMI_WITH_DISASS_OVERFLOW,DBG_EVENTS

ccrause

  • Hero Member
  • *****
  • Posts: 845
Is it correct for avr that the exe is loaded at very low addressed? 0x011a ?
Yes, see attached disassembled source (project1_avr.lss.zip) for memory addresses.

Also attached a debug log of a Lazarus debugging session for AVR (lazarus_avr.log.zip). The only thing that seems strange is that after the -exec-continue command, there appears to be an extra "(gdb)" in the output.  See from around line 1003 in the log:
Code: [Select]
  DebuggerState: Finished dsRun
  << TCmdLineDebugger.ReadLn "(gdb) "
  ERROR: Got NO stop params at all, but was running
The (what I think) is the expected output << TCmdLineDebugger.ReadLn "=breakpoint-modified,bkpt= only gets read on line 1057.

A similar debug test with target Linux reads the breakpoint message immediately after issuing the -exec-continue command:
Code: [Select]
  DebuggerState: Finished dsRun
  << TCmdLineDebugger.ReadLn "=breakpoint-modified,bkpt=

Could this perhaps mean that avr-gdb is emmitting and extra (gdb) line of text that isn't expected and should be ignored?

ps. Thanks for the link to the wiki page for debugger tips Martin.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9792
  • Debugger - SynEdit - and more
    • wiki
The extra gdb prompt is normal.

Remote debug uses gdb in async mode.
Normally if the exe in gdb is running, gdb does not accept any commands until the exe stops/pauses.
But over remote this is necessary in order to send an "interrupt" (pause request) command. (In none remote, the IDE itself (bypassing gdb) sends a signal to the exe).

In Async mode after a run command, the 1st "(gdb)" prompt acknowledges that the run command was accepted.
The 2nd "(gdb)" prompt is sent, when the exe is stopped/paused.

-----------
The problem in your log is, the order in which data comes from the debugger.

The IDE expects the "*stopped,reason=" before the prompt. (And that may be wrong of the IDE).
In your example the "(gdb)" prompt happens first, the "*stopped,reason=" later (and therefore ignored).

So the IDE does not know where the app stopped, and that explains the lack of location and asm.
(You may be able to get asm, by right clicking a stackframe in the stack win, and choose disasm from the context menu)

Please report a bug:
"
remote debug fails to get break location, if gdb sends "*stopped,reason="  late.
See http://forum.lazarus-ide.org/index.php/topic,37405.msg252081.html#msg252081
"
And attach the log.


------------------------
I have no clue what causes the wrong watch value.
Have you tried stabs/dwarf?
And -O-

Otherwise I suggest to post a question on the fpc mail list.

« Last Edit: July 09, 2017, 08:15:21 pm by Martin_fr »

ccrause

  • Hero Member
  • *****
  • Posts: 845
Bug report: issue 0032130

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9792
  • Debugger - SynEdit - and more
    • wiki
Could you test? See comment on bug report.

ccrause

  • Hero Member
  • *****
  • Posts: 845
I've tested r.55619 with RequirePID patch from 32084.  It seems as if the IDE now stops and highlights the break point, but then a whole string of disassembler instructions gets generated without end.  I have to stop the debugger to stop the dissembler instruction flood.  From the Lazarus log output the following line seems to  indicate the start of attempted disassembly on line 907 (see attachment):
Code: [Select]
INFO: TDBGDisassembler.FindRange: Address not found 282 wanted-before=15 wanted-after=17 in map with count=0

Address 282 is the address of the assembler instructions for the line of code at the break point.  A bit further down the log the following seems suspicious:
Code: [Select]
    >> TCmdLineDebugger.SendCmdLn "-data-disassemble -s 42 -e 466 -- 0"
    << TCmdLineDebugger.ReadLn "^done,asm_insns=[{address="0x0080002a",inst="nop"},{address="0x0080002c",inst="nop"},

It seems as if avr-gdb added 0x800000 to the address, perhaps this is caused by this known issue?  I will investigate further at a later stage.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9792
  • Debugger - SynEdit - and more
    • wiki
good news for the breakpoint.

About the dis-assembler:

there are 2 issues (from your description / I will need to find time to dive through the log later)

1) gdb adding 0x80000
2) the IDE trying forever.

1) is a gdb issue, and probably not solvable from the IDE.
It may also lead to the IDE not understanding the result, and discarding it.  (that is unlikely to get fixed in the IDE)

2) this should be fixed in the IDE. the IDE should detect it is not getting sensible data, and should stop.


-----------------------
In general the disasm code of the IDE is complicated.

Because the IDE attempts to show disass code from after and BEFORE the current location.

Disassembling from the current location is easy. The address is known.

But disassembling from before is tricky.
The byte length of each asm statement is different. That means there is no way of knowing if current_location -0x10 is the begin of a statement or not.
But even if it is not, you can start disassembling there, but you get wrong code.

So the IDE tries to look for begin of a function, or it tries from different locations and uses some heuristics.
That is probably where it currently ends up in a loop.

Michael Collier

  • Sr. Member
  • ****
  • Posts: 301
I was trying out the avr-gdb command line debugger from within a windows shell, I asked avr-gdb for the memory location of global variable (it gave 0x800200 which I thought looked strange). When I peeked in the address I received error. Here is shell output:

Code: DOS  [Select][+][-]
  1. (gdb) p &my_global
  2. $3 = (int *) 0x800200
  3. (gdb) x 800200
  4. 0xc35c8:        Ignoring packet error, continuing...

I googled for "0x800000 atmel avr memory" and some pages were found striaght away:
http://www.atmel.com/webdoc/avrlibcreferencemanual/FAQ_1faq_ramoverlap.html
http://www.atmel.com/webdoc/avrlibcreferencemanual/mem_sections_1sec_dot_data.html

Note: As I am writing this I got notification of new post, so maybe martin_fr can help better, hope this helps..


ccrause

  • Hero Member
  • *****
  • Posts: 845
1) is a gdb issue, and probably not solvable from the IDE.
It may also lead to the IDE not understanding the result, and discarding it.  (that is unlikely to get fixed in the IDE)
I'm busy compiling gdb 8.0 with this patch for the avr_integer_to_address internal function which should correct the 0x800000 offset in program memory.  Will test again once I get gdb running.

ccrause

  • Hero Member
  • *****
  • Posts: 845
I think the first issue of detecting that gdb stopped is fixed.  Using a patched gdb the disassembling works and Lazarus displays the disassembled code correctly.  Will log a separate issue to stop the disassemble process eventually to cater for the normal behaviour of gdb for AVR.

Thanks Martin!

 

TinyPortal © 2005-2018