Memory Manager

[Thaddy]

Also note that most FPC memory managers already have some more information (compared to - say - Delphi) at negative offset, because the actual allocated block size is stored, not the bucket size. There is no reason not to extend such information if required. See the example code for the mechanism.

[damieiro]

@taazz.

It's exactly what you say. I have no trouble with my own code. I use my own interface then i call the memory manager. Really i catch on the function and account p in other ways in my own code. But i can't do it with RTL or any compiler defaults. Same for 3rd party.

But i cannot "catch" the RTL calls to memory manager with p because their entry point has no p in the interface.

@Aserge. As it's said.  RTL/other programs calls the memorymanager.getmen (size) not memorymanager.realloc (p,size). And getmen and realloc, allthough similar, aren't the same operation.

@thaddy. Yes a memory manager is not easy. But this is not for difficulty or easines, it's to have the tool to make and test it not only in my code. And yes, there is on negative offsets (p-8 and p-16 in fpc in 64 bits -> 8=sizeof(ptruint)) some pointer information. I would like to extend this too

[marcov]

You can't anyway, since getmem is defined OUT.

Moreover, even if it weren't, it might be a random value(e.g. newly defined uninitialized variable), so unless you can make sure it is a valid, allocated block by looking it up in the heapmanager's internals (*), touching it will only crash.

This is WHY it is defined out 

(*) which I doubt.

[damieiro]

It's more easy.

p is defined out because we want. No other reason.

A memory manager CAN know if a p was given by the memory manager or not. There are a lot of tecniques. Examples.

1.- Mark at some offset (for example p-24) a signature for the memory manager or some CRC.
2.- Have a list of returned p's. If not in your list, then that p is not yours.
3.- Other enforcement methods (allways nil a dispose pointer...)

If one can read p, you can know if it has passed by the memory manager. Even if your memory manager then calls the default sysgetmen. One thing is accounting what are you doing with your pointers (if they are alive, coherent, etc)  and a different thing is how the memory is reserved or heap managed. Two separate problems.

[PascalDragon]

No, you can't say for sure, because p could contain a garbage value that by chance is the same value as a valid pointer allocated by the heap manager.

[marcov]

No, you can't say for sure, because p could contain a garbage value that by chance is the same value as a valid pointer allocated by the heap manager.

Moreover, even if it is valid, it is not guaranteed the only one (there might be copies), so it is useless for GC schemes.

[damieiro]

Hey. Even a realloc (p,size), p could be anything. 

That's not the point. A memory manager can recognize a pointer it gives by itself. And no, a memory manager cannot give addresses to stack zone, only heap zone, so any p not referencing any in heap it's detectable. And even if two memory managers are working (like a cmanager for a dll) you can mark your references.

We can think in pointer manipulation. And with a memory manager you could extend the syntax. For example Copypointer (p1,p2). But i insist, this is not the point too.

I think you are mainly thinking in Garbage Collector perhaps by some examples i've put. No, that's not (too) the point i try to explain (i'm sorry for my bad english). It can be used for many other things.

My main use it's not GC. It's debugging and teaching  . I can see better bad pointer usage (by any mistake or in a way we do not like) knowing p caller, specially with classes.  Even if the memory manager does nothing different with memory allocation, only bookeeping. Or even enforcing programming styles. If i want to enforce a nil for non used pointers, i will like that my bookeeper memory manager process take note of with p's processes in In and Out. Even i could have these functionality with conditional compiler directives if well designed and normal mode if not compiler directive present.

The main issue (This is The Point), is that i am saying that actually the entry is a function and should be a procedure with p or a function with p in somewhere visible to know all variables for the memory manager to perform any actions (whatever the programmer desires) and knowing it in time it helps warn/bug/do things.

So if the p data is useful, then the data should be known -> THE POINT

And for 3rd party code and (mainly) the RTL i cannot see it because the function entry mask us the data. The RTL calls the memory manager and i only know what p is given, not what p is demanding (well, in classes i can do some dirty tricks with class constructors and create/after create methods, but these are other things not needed if you can see calling p and bad for elegant code...)

So, please, do not discuss about use. I'm pretty convinced it's useful. Or in the worst case, if you are not convinced (or i'm in error), never harmful (it can be ignored for all existing code, it's a feature for new code with can use it or ignoring it as now).

The solution is simple as hell:
When I write GetMem(P, Size); or P := GetMem(Size); compiler inserts a call to MemoryManager.GetMem(P,Size) and not P:=MemoryManager.GetMem(Size) as is doing now.
When compiler does RTL things (like new p, class instantiation), call the way above.

In Memory Manager Record, add the new calling method with redirecting code (In procedure memorymanager.getmen (p,size) call the old function entry as default implementation) and go. It should go ok without performance issues with compiler optimization. And if the new calling method (the procedural way) is intercepted, then it could be used to do some cool things.

What is the problem with this way? Old code should go ok in every case (even memory manager code like heaptrc) and the only thing i see is minor performance hit if compiler can't optimice it (and if compiler can't optimice a straight forward call, then i think it should be viewed why).
And if performance were an issue, a simple compiler directive (Let's be creative : $MemoryCallMethod) could do the task (if no directive all calls by old way, if new directive, with procedural way... that will affect only to non default memory manager code that can be bridged as posted)

And it even could be useful to see things that i do not like as pointer dealocation differences by platform.. (example disposeof here if i read it correctly-> http://forum.lazarus.freepascal.org/index.php/topic,42398.0.html)

So i do not see any bad thing, but some promissing good ones. Who said fear? 

[marcov]

What is the problem with this way?

- the memory manager record/structure is runtime, not compile time.
- the layout of the memory manager record is delphi compatible, changing it makes it incompatible.

[damieiro]

- the memory manager record/structure is runtime, not compile time.

I do not understand this. Please explain more. It think it will work the same. Why it wouldn't work?.  I need that compiler changes their entry point to this structure.

- the layout of the memory manager record is delphi compatible, changing it makes it incompatible.

Why this solution won't work on delphi (or other) code?

When I write GetMem(P, Size); or P := GetMem(Size); compiler inserts a call to MemoryManager.GetMem(P,Size) and not P:=MemoryManager.GetMem(Size) as is doing now.
When compiler does RTL things (like new p, class instantiation), call the way above.

In Memory Manager Record, add the new calling method with redirecting code (In procedure memorymanager.getmen (p,size) call the old function entry as default implementation) and go.

Code: Pascal  [Select][+][-]

1. {pseudocode}
2. MemoryManager.GetMem (var P:pointer; size: ptruint);
3. begin
4.   P:=MemoryManager.GetMem (size);
5. end;
6.  
7.  

Or even why should not work having p "visible" in other zone if redirect weren't used (like Result) ?. Please, explain.