F8 acts like F9

[440bx]

Hello,

In one case when stepping (F8) on an instruction, instead of having that single instruction executed, the debugger's behavior is like F9 (Run) had been pressed instead.  F7 works fine.

Attached is the project that exhibits the problem.

The screenshot shows where the execution first stops (line 101), on that line, pressing f7 works as expected but, pressing f8 instead of executing a single instruction, executes the entire program.

Thank you for your help.

[Martin_fr]

You are allocating memory on the stack?
So you are changing the stack pointer?


The debugger needs to skip the call. When the call returns, the debugger checks if it is back at the correct stack. And that probably fails. The debugger probably thinks it is in a recursive call of the current function.

Your code will have an asm "call ..."
The debugger set a break after that.
But it is not enough to reach that breakpoint => if the function enters recursive, then it needs to be at the break in the current stackframe.

Not sure how to handle that best. There is EBP (base pointer) but not all code uses it. So it isn't reliable (it is within fpc generated code).

Does gdb handle this?

[440bx]

You are allocating memory on the stack?
So you are changing the stack pointer?

Yes on both counts.  esp is modified (in order to allocate stack memory), ebp is left untouched (no reason to mess with it.)

The debugger needs to skip the call. When the call returns, the debugger checks if it is back at the correct stack. And that probably fails. The debugger probably thinks it is in a recursive call of the current function.

Your code will have an asm "call ..."
The debugger set a break after that.
But it is not enough to reach that breakpoint => if the function enters recursive, then it needs to be at the break in the current stackframe.

Not sure how to handle that best. There is EBP (base pointer) but not all code uses it. So it isn't reliable (it is within fpc generated code).

Does gdb handle this?

I tried GDB and it handled it normally.  IOW, it stepped the call just like any other call.

[Martin_fr]

I wonder how gdb does it (though not going to find out / the gdb sources are to much work to go through).

Maybe gdb analyzes the asm instructions in the called function. It does that in some other cases). And since c compilers have such a function (though I would imagine c compilers will inline such code?) it might recognize the pattern.
=> If gdb does that, then you are just lucky that your code is similar enough to whatever patterns gdb knows. (i.e whatever c compilers generate for this).
=> If gdb does it some other way... I have no idea.


The problem is to find a fool proof way of establishing that the code is back to the original frame.

In all other cases that I can think of, if the EIP is at
   call ....
- then call will push the return address.
- the callee will have to have the eriginal ESP in order to execute the "ret" statement. Or it wont see the return address.

You code must be fiddling with were the return address is. Otherwise it couldn't get back to the caller?

[440bx]

The problem is to find a fool proof way of establishing that the code is back to the original frame.

The one thing that comes to mind is that after the stack allocation, EBP/RBP _must_ have the value they had before the call.  IOW, ESP/RSP may change due to stack memory being allocated but, EBP/RBP _must_ stay the same.

As far as the code sequence being similar to what C compilers do, it is identical.  In this case, it would be MSVC and, the call consists of placing the number of bytes to allocate, followed by the call to _alloc_probe.

- the callee will have to have the eriginal ESP in order to execute the "ret" statement. Or it wont see the return address.

You code must be fiddling with were the return address is. Otherwise it couldn't get back to the caller?

The allocation behaves like a local variable.  IOW, it is valid _only_ in the function/procedure in which it was allocated.  My code does it the simplest way which is, restore ESP/RSP to its original value before exiting the function.  That way, ESP/RSP has the value originally set by the compiler which means that when the stack frame is taken down, the stack pointer points to the right place because the stack allocation has been "freed" (by adding the number of bytes allocated back to ESP/RSP.)  That way when EBP/RBP is popped, RSP is pointing to the original EBP/RBP value (which is what the compiler expects.)

[Martin_fr]

The problem is to find a fool proof way of establishing that the code is back to the original frame.

The one thing that comes to mind is that after the stack allocation, EBP/RBP _must_ have the value they had before the call.  IOW, ESP/RSP may change due to stack memory being allocated but, EBP/RBP _must_ stay the same.

**IF** the basepointer is used at all... (Some code does not use it, and also not set it / That is why trying to get the stack sometimes fails, when only relying on BP)

Well, to be exact, the value of BP will be the same after the call, even if it is not used. But that is meaningless because in that case BP will also be the same inside a recursion. And that would mean that, if the call was recursive, the "step over" would stop inside the recursive call.

Of course maybe there is a way to find out if the calling function (the function in which the step-over starts) is using the BP. Then it could be used based on the result of such a check. Not sure, it may be avail in DWARF CFI info.... but that will need some (quite some) read up. Or parsing the callers asm at function start.

The allocation behaves like a local variable.

I know the general concept. Though not the different ways it may be implemented.

  IOW, it is valid _only_ in the function/procedure in which it was allocated.  My code does it the simplest way which is, restore ESP/RSP to its original value before exiting the function.

Which function? the function that did the alloc? or the function that called the alloc?

The latter shouldn't need any work when exiting? the "leave" instruction (or restore from BP) should do it without any extra code needed?

That is of course if the function has a leave statement.

If you compile code with (IIRC)
{$Optimization FORCENOSTACKFRAME}
then that may not happen (though, I am not sure, if FPC will honour that for procs that call other procs...)

That way, ESP/RSP has the value originally set by the compiler which means that when the stack frame is taken down, the stack pointer points to the right place because the stack allocation has been "freed" (by adding the number of bytes allocated back to ESP/RSP.)  That way when EBP/RBP is popped, RSP is pointing to the original EBP/RBP value (which is what the compiler expects.)

Did you swap sp <> bp in this statement?

Or is that a reference to when the function in which you tried to "step over" will exit, rather than when the function over which you tried to step returns?

SP is restored based on  BP, BP holds the original SP. (after it itself was pushed)
But SP then points to the end of allocation, so allocation more must change SP.




Fixing this will probably take some time (before I get to it).

Also, this will require me to test with c compilers.
As this will add a cost (slight slow down) to step over, this is not going to be done based on your personal code only (and then affecting everyone).
This should be done, so it works with code everyone may encounter while stepping through c compiled code. (which fpdebug should eventually be able to do, as fpdebug is a generic debugger).

If at that point I have your code available it can probably be part of the recognized set of conditions for this.

If your code can be published, I would recommend to create a bug report with a full self contained example (including the alloc code).

But as I said: someday later...

[440bx]

Which function? the function that did the alloc? or the function that called the alloc?

The function that did the allocation.

By the way, I posted the project's code in the OP.  It's a tiny program and you can see exactly what's it's doing, how and when.  It's one of my many test programs to "exercise" ntdll APIs.

Did you swap sp <> bp in this statement?

No, ESP/RSP is restored before the function exits to ensure the value is that expected by the compiler (in this case FPC.)

Also, I figured I'd report the fact that fpdebug didn't behave as expected in that specific case.  The "problem" is of little consequence as it is exceptionally rare that I need to dynamically allocate stack space in a function/procedure.

Lastly, since the function is in ntdll, f7 steps over the call (just as f8 normally does) and, f7 operates as expected.  My point here is that, in this particular case - calling external code for which there is no source - f7 and f8 should be the same, shouldn't they ?  therefore, whatever f7 does (which is correct), f8 should do in this case.
 

[Martin_fr]

Found it.

Didn't realize it was a kernel function. I thought you wrote your own.

[440bx]

Found it.

Didn't realize it was a kernel function. I thought you wrote your own.

No problem. 

It's a bit of a hassle to write that kind of code because it cannot use a stack frame since it is allocating space on the stack.  In addition to that, it differs depending on the program's bitness. That's why the function takes its parameter in eax (32bit)/rax (64bit).  Anyway...

I hope the little program will be useful in figuring things out.

[Martin_fr]

Well please report a bug anyway, so it wont get forgotten.

It probably is possible to check from dwarf info if the BP is in use at a given address. If so, then that can be used to check the frame.

[440bx]

Filed a bug report.

Link:   https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/41649
ticket: 41649