Feedback request: Syntax for "intrinsic functions" in FpDebug

[Martin_fr]

Btw, the forum has [ quote ]... [ / quote ] blocks....

> At least for strings and dyn array, the location of the data in memory is known, and can be read.

Yes, but when Delphi/FPC passes managed type into a function as non-const parameter, it already increases counter.
I exactly meant that this should be prevented.

IIRC, in FPC the receiving routine does the refcount handling.
So if you have a watch "pos(var1, var2)" then "pos" will inc and dec the refcount. Nothing that can be done there.
If var2 has a refcount of 0 before the call (due to some severe bug) then yes, such a watch will free the memory of var2.  But it's the called function, not the debugger that does it.

You can easily find it. But internal details is not hat i pointed at, i pointed exactly at very non-expected behavior, which only makes sense when you draft possible implementations. And for programmer who wants to just use the library - it is a hidden trap. And hidden funciton calls by watches accessing properties can trigger it.

Big first point on https://wiki.freepascal.org/FpDebug-Watches-FunctionEval : function eval can have side effects.

I remember too well, my app always crashed. Except when I was debugging it. Because back then I had no idea the watch was creating the missing data in the target app.

A watch could even make calls to the OS. Teaching the debugger how to revert the effects of every possible call to the kernel or any library => nope.

> So That means however, if you were to eval a watch, that calls a function, and that function returns an interface => then that refcount goes up by one, and never down (because it waits for the caller to "release" the interface).

But what is the caller here? i think the caller is exactly the evaluating functions. As soon as the value was rendered to the screen - the functin exist, and its local variables (including that nterface pointer) are released. Evaluating naturally includes changing refcounter forth and back.

The debugger is the caller, if the function result is of type (managed) interface.

> But that does not work now, because fpc does not provide enough info for the debugger to do so.

I'll have to think about it. At least in Delphi that is a nonsense. The ony thing needed ot know is "this is interface type", then COM standard dictates how exactly references are released, ABI is fixed forever.
The only thing debugger/evaluatoir has to know for being able to free is "the data type is some interface", it would be enough.

If it is a com interface....
But that must be known.

And IIRC, fpc does not put that into the DWARF info. (sadly)

[Arioch]

> Forum has [ quote ]... [ / quote ] blocks.

I know , but they are much longer to type (or require a third arm for mouse, while other two arms are on keyboard.)

> If var2 has a refcount of 0 before the call (due to some severe bug) then yes, such a watch will free the memory of var2.  But it's the called function, not the debugger that does it.

Frankly, it is mostly an argument about definitions, funny but usually meaning little. If the program died with division by zero error, was it an error in the program or an error in the function but not in the program? If debugger calls the function that then unexpectedly destroys something - then from user point of view it was destroyed by debugger. Just like a prey is killed by the hunter, not by the bullet or by the rifle.

If we go really techy, then the function did nothing wrong, it was inconsistent state of the passed parameter object that led to disaster. And then who was the actor, who gave that inconsistent value to the function? it was the watch window. On the next iteration it would be unsavvy devleloper, who enabled function calls in watch window without making sure the function works safely on incorrect data. And so forth go blame games.

From appliaction programmer's point of view, function is the code he types. Programmer does not do refcounting, it is "Delphi" or "Lazarus" which does it automatically. Compiler developer then would shout "learn your tools". AppDev would yell "fix your tools" and so it goes, making the question "who did it" senseless.

> of 0 before the call (due to some severe bug)

The thing is, the programmer (the program he writes) did not do any calls yet, he is tracing through the object nested constructors (bonus points for foing it in assembler window not even Pascal one). It is debugger then which suddenly calls a funciton (like property getter) before the object is ready for it.

So there is "severe bug" in the program, there was not enough of understanding of inner working of the debugger. Or not enough of forethought and sharp memory.

> Teaching the debugger how to revert the effects of every possible call to the kernel or any library => nope

Not in Pascal, yes. It would need a different language.

But implementing intrinsics in the way that would preclude side effects, and then not extending in the ways breaking it  - is possible. That is what i wrote. Special care has to be done that intrinsics would be free from side effects by design and by implementation.

Otherwise, debugger's :length is worse than RTL's Length. But if an intrinsic gives warranty of never triggering side effects (even on inconsistent program states and data structures), then it alone makes a good reason to duplicate things. I think it shuld be requirement for current and futue intrinsics to never ever have side effects, better fail than this.

> If it is a com interface....  But that must be known.

In Delphi Classic it is by definition so. But as soon as we go cross-platform - it can no more be. At most it cold have been Mozilla XPCOM, but i guess it also does not exist on every FPC-supported platform. So it is even less certainty in FPC than there is in Delphi/COM (where it is lacking already).

> And IIRC, fpc does not put that into the DWARF info

I don't know DWARF, but i also would not be sure FPC even could put that info there without breaking compatibility with some "old classic" tools.
Not all formats are extensible enough, and when they are - practical programs are not always forgiving...

[PascalDragon]

> But that does not work now, because fpc does not provide enough info for the debugger to do so.

I'll have to think about it. At least in Delphi that is a nonsense. The ony thing needed ot know is "this is interface type", then COM standard dictates how exactly references are released, ABI is fixed forever.
The only thing debugger/evaluatoir has to know for being able to free is "the data type is some interface", it would be enough.

For interfaces in general you can not know the reference count, because IUnknown only provides methods to increment (_AddRef) and decrement (_Release) the reference count, but not to retrieve it. And doing a _AddRef followed by a _Release (both of which will return the reference count) might be a rather involved operation depending on the implementation so it might not be a good idea to do that inside the debugger.

[Martin_fr]

And doing a _AddRef followed by a _Release (both of which will return the reference count) might be a rather involved operation depending on the implementation so it might not be a good idea to do that inside the debugger.

Also FPC currently doesn't even supply the addresses of those routines.

The dwarf entry for any interface currently is an address, the typename, and "this is a structure". And nothing more. No names or addresses of routines. Nothing.

[Arioch]

For interfaces in general you can not know the reference count, because IUnknown only provides methods...

Yes, and i spelled it above. Incapsulation agaisnt introspeciton :-)

And doing a _AddRef followed by a _Release

is exactly what we try to avoid introducing non-destructive intrinsics.

So, yes, in general case it would be impossible, only if we can somehow learn some well-known implementation like TInterfacedObject was used

[Arioch]

The dwarf entry for any interface currently is an address, the typename, and "this is a structure". And nothing more. No names or addresses of routines. Nothing.

address of what?

COM ABI mandates fixed positions for those methods at fixed offsets from every "interface pointer".
That is the only way interfaces could work, since inheritance concept was damned by COM.

similar how TObject provides some virtual functions that are now forever at fixed offsets of any TClass pointer.

[marcov]

address of what?

COM ABI mandates fixed positions for those methods at fixed offsets from every "interface pointer".
That is the only way interfaces could work, since inheritance concept was damned by COM.

Freepascal also allows non-COM interfaces (as parent of  IUnknown)

[Arioch]

address of what?

COM ABI mandates fixed positions for those methods at fixed offsets from every "interface pointer".
That is the only way interfaces could work, since inheritance concept was damned by COM.

Freepascal also allows non-COM interfaces (as parent of  IUnknown)

Then perhaps FPC can extend DWARF with signalling if it was IUnknown (wih children) or non-IUnknown interface?

P.S. there was a significant holy war within Firebird 3 and their new API, if IUnknown is COM or on top of COM.
It seems that from rigorous point of view COM does not require IUnknown and hence does not include those three key methods.
But in practical use of terms, COM and IUnknown are siamese brothers,

[Martin_fr]

Then perhaps FPC can extend DWARF with signalling if it was IUnknown (wih children) or non-IUnknown interface?

"IUnknown" simple means that there are Inc/Decref methods. It does not tell us anything about where that data is. In fact I could implement that interface in my own class, and since I plan one global instance that will be for the lifetime of the app, I can get away with not even storing the refcount.

And then even FPC wouldn't know where the ref count is stored (or if it is actually stored at all). It couldn't give that info to the debugger, and the debugger can't know either.

FPC should however add
- inheritance info
- names and location of methods

And because a debugger can be allowed to know more than the app, FPC could add info how to find the underlaying object (if exists). Though this part is particularly tricky...

Actually, the debugger could already find that object, but it would have to disassemble code, and it would need to find exactly the code it expects. I.e. it would need have code specific to different fpc versions, or fpc on different platforms, ..... Would still not get the refcount ....

[Arioch]

Then perhaps FPC can extend DWARF with signalling if it was IUnknown (wih children) or non-IUnknown interface?

"IUnknown" simple means that there are Inc/Decref methods. It does not tell us anything about where that data is

Sure. I did not say you would get access to the counter. I said long ago COM/IUnknown does not expose it and that in general case peeking into it would not be possible. It is kind of becoming running joke, we repeat it to one another again and again :-)

Here i was thinking just about enabling evaluation/inspection, which is actually more dependent upon the 3rd methopd, QueryInterface.
Actually, AFAIR that was exactly how Delphi implemented typecasts back to classes. Priorly all kinds of hacks were used, and eventualyl Delphi just made a special interface to return TObject pointer.

So two topics - general detection of IUnknown and specifically non-destructive intrinsics got mixed in and confused.

> In fact I could implement that interface in my own class,

I know. It is typical thing to, for example, disable actual reference counting (then no counter at all), when the interface used to access, for example, DOM nodes of XML or JSON tree.
And other purposes too.

Actually, the debugger could already find that object, but it would have to disassemble code, and it would need to find exactly the code it expects. I.e. it would need have code specific to different fpc versions, or fpc on different platforms, ..... Would still not get the refcount ....

Veeery hackish way. Even Delphi was not going THAT deep to get from interfaces back to objects AFAIr.

That said, is it practical scenario that Lazarus is used to debug a program with no Pascal sources?
Because if you have sources - IDE would probably recompile them compile with the same RTL as Lazarus itself and debugger use. And then no need to have a database of different versions.

There are a number of stand-alone debuggers, and Lazarus is not one of them.

However, this still seems badly fragile.
Like hackish functions trying to understand if a random pointer is object or not. Most time they work good, other time they damage program, like burning stack guard pages

[Martin_fr]

An idea for intrinsic - watch (non-destructively) references counter for objects, interfaces, strings and dynarrays.

Done for AnsiString(dwarf-3) and DynArrays

https://wiki.freepascal.org/FpDebug-Watches-Intrinsic-Functions