Recent

Author Topic: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)  (Read 2448 times)

Avinash

  • Full Member
  • ***
  • Posts: 126
Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« on: July 23, 2023, 07:13:06 pm »
Code: Pascal  [Select][+][-]
  1. var
  2.   I: Integer;
  3.   P: Pointer;
  4. begin
  5.  
  6.   for I := 1 to 10 do
  7.   begin
  8.     Write(GetFPCHeapStatus.CurrHeapUsed,' => ');
  9.     GetMem(P, 0);
  10.     FreeMem(P, 0);
  11.     WriteLn(GetFPCHeapStatus.CurrHeapUsed);
  12.   end;
  13.  
  14. end.

Output:

112 => 128
128 => 144
144 => 160
160 => 176
176 => 192
192 => 208
208 => 224
224 => 240
240 => 256
256 => 272


FPC 3.2.2 Win32

jamie

  • Hero Member
  • *****
  • Posts: 7753
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #1 on: July 23, 2023, 07:23:54 pm »
did you use heaptrc to confirm that?
The only true wisdom is knowing you know nothing

runewalsh

  • Full Member
  • ***
  • Posts: 121
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #2 on: July 23, 2023, 07:34:01 pm »
Made a patch, vote up, follow me on Twitch & Patreon, buy my merch.

Avinash

  • Full Member
  • ***
  • Posts: 126
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #3 on: July 23, 2023, 07:41:49 pm »
did you use heaptrc to confirm that?
No, never used. It's not about the title and the right words, it's about the fact shown. If a non-zero value is used, then there is no leakage of 16 bytes per round.

Curt Carpenter

  • Hero Member
  • *****
  • Posts: 756
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #4 on: July 23, 2023, 09:01:11 pm »
It's an interesting corner case that I can see might come up as a practical matter if one were executing something like GetMem(P,BlockSize) and FreeMem(P,Blocksize) where the variable BlockSize might possibly be set to 0 at some point.

jamie

  • Hero Member
  • *****
  • Posts: 7753
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #5 on: July 23, 2023, 09:09:07 pm »
I just did a test case using the heaptrc and it states there are no unfree blocks of memory.

This only tells me the memory manager is smart enough to reuse those cases.

So what you are saying is you think you can fill up the table with empty descriptors?
« Last Edit: July 23, 2023, 09:10:55 pm by jamie »
The only true wisdom is knowing you know nothing

Avinash

  • Full Member
  • ***
  • Posts: 126
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #6 on: July 23, 2023, 09:33:07 pm »
I just did a test case using the heaptrc and it states there are no unfree blocks of memory.
Because the bug is not in my program, bug in FPC.

jamie

  • Hero Member
  • *****
  • Posts: 7753
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #7 on: July 23, 2023, 09:43:30 pm »
If you say so, I don't think you got the part where the manager reuses the items.

But whatever.
 
The only true wisdom is knowing you know nothing

Avinash

  • Full Member
  • ***
  • Posts: 126
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #8 on: July 23, 2023, 10:08:27 pm »
the part where the manager reuses the items.
Code: Pascal  [Select][+][-]
  1. const Mb = 100;
  2. var
  3.   I: LongInt;
  4.   P: Pointer;
  5. begin
  6.  
  7.   I := 0;
  8.   Write(GetFPCHeapStatus.CurrHeapUsed,' bytes => ');
  9.   repeat
  10.     GetMem(P, 0);
  11.     FreeMem(P, 0);
  12.     Inc(I);
  13.   until GetFPCHeapStatus.CurrHeapUsed >= Mb*sqr(1024);
  14.   WriteLn(GetFPCHeapStatus.CurrHeapUsed/sqr(1024):0:1,' megabytes in ',I,' cycles');
  15.   ReadLn;
  16.  
  17. end.

check memory usage in task manager

Bart

  • Hero Member
  • *****
  • Posts: 5727
    • Bart en Mariska's Webstek

Warfley

  • Hero Member
  • *****
  • Posts: 2066
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #10 on: July 24, 2023, 11:13:14 am »
did you use heaptrc to confirm that?
As this is a problem in the memory manager and heaptrc is registering a custom memory manager, this may not actually measure the same thing.
check memory usage in task manager
First, never trust your task manager about memory usage:
Code: Pascal  [Select][+][-]
  1. var
  2.   p: Pointer;
  3. begin
  4.   p := Getmem(2000000000);
  5.   ReadLn;
  6. end.
I've just allocated 2 gigabytes of memory but my task manager tells me that my program has only 1.8 megs.

Secondly, as jamie pointed out, the memory manager may not reuse previously allocated memory:
Code: Pascal  [Select][+][-]
  1. uses
  2.   heaptrc;
  3.  
  4. var
  5.   i: Integer;
  6.   p: PInteger;
  7. begin
  8.   heaptrc.keepreleased := True; // Override reusing behavior
  9.   Write(GetFPCHeapStatus.CurrHeapUsed,' bytes => ');
  10.   for i := 0 to 10000 do
  11.   begin
  12.     new(p);
  13.     dispose(p);
  14.   end;
  15.   WriteLn(GetFPCHeapStatus.CurrHeapUsed);
  16. end.
The memory manager can decide when and where to reuse memory from previous allocations. The example above uses the feature of heaptrc to completely disable any reuse, this is a special tool that allows easier detecting of use-after-free. No reuse allocation methods are also often used in security critical software like the sandboxing of browsers to ensure use-after-free cannot happen. Thats why browser alaways seem to be so memory hungry, it's simply because they make massive use of virtual memory and paging instead of reling on address reuse.

Note that while this above is of course forced through keepreleased of heaptrc, also the default memory manager does not always reuse memory. The reason for this is that the FPC memory manager is unlike the C memory manager not a first come first serve memory manager, but has different allocation lists for different sizes to avoid fragmentation:
Code: Pascal  [Select][+][-]
  1. var
  2.   p: PInteger;
  3.   sl: TStringList;
  4. begin
  5.   Write(GetFPCHeapStatus.CurrHeapUsed,' bytes => ');
  6.   new(p);
  7.   dispose(p);
  8.   sl := TStringList.Create;
  9.   sl.Free;
  10.   WriteLn(GetFPCHeapStatus.CurrHeapUsed);
  11. end.
Result:
Code: Text  [Select][+][-]
  1. 4416 bytes => 4512
As you can see it will keep some blocks allocated. The reason for this is that it allocates blocks for size 4 (Integers), as well as blocks for TStringList (and it's underlying string arrays), and keeps it around for those blocks, instead of just reuising the same blocks or freeing them completely.

As comparison see what happens when you use the C memory manager instead:
Code: Pascal  [Select][+][-]
  1. uses
  2.   cmem,
  3.   Classes;
  4.  
  5. var
  6.   p: PInteger;
  7.   sl: TStringList;
  8. begin
  9.   Write(GetFPCHeapStatus.CurrHeapUsed,' bytes => ');
  10.   new(p);
  11.   dispose(p);
  12.   sl := TStringList.Create;
  13.   sl.Free;
  14.   WriteLn(GetFPCHeapStatus.CurrHeapUsed);  
  15. end.
Result:
Code: Pascal  [Select][+][-]
  1. 0 bytes => 0
This memory manager now does uses a first come first serve allocation method, meaning that instead of like before having two blocks allocated for different sizes and being kept around for the next allocations, now there is just one block which will be freed afterwards.

Neither of these two examples produces a memory leak.

Long story short, never make assumptions about memory leaks based on such simplified information like the allocation state. This is heaviely unreliable as it does not account for things like reuse policy, different allocation strategies, etc.

If you want to check for memory leaks, instead use tools like heaptrc (even though in this case it is not of much use as it overrides the memory manager), or valgrind

Avinash

  • Full Member
  • ***
  • Posts: 126
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #11 on: July 24, 2023, 11:50:14 am »
Long story short, never make assumptions about memory leaks based on such simplified information like the allocation state.
It is not so easy to refute something that has been blessed by decades of empirical experience.

I've just allocated 2 gigabytes of memory but my task manager tells me that my program has only 1.8 megs.
The correct counterexample would be to demonstrate when the task manager shows that the memory is used, but in fact it is not.

Code: Text  [Select][+][-]
  1. 4416 bytes => 4512
As you can see it will keep some blocks allocated.

Common truth, but as soon as it is wrapped in a loop, the situation immediately becomes clear:
Code: Text  [Select][+][-]
  1. 2464 bytes => 2512
  2. 2512 bytes => 2512
  3. 2512 bytes => 2512
  4. 2512 bytes => 2512
  5. 2512 bytes => 2512
  6. 2512 bytes => 2512
  7. 2512 bytes => 2512
  8. 2512 bytes => 2512
  9. 2512 bytes => 2512
  10. 2512 bytes => 2512

Warfley

  • Hero Member
  • *****
  • Posts: 2066
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #12 on: July 24, 2023, 12:06:38 pm »
Common truth, but as soon as it is wrapped in a loop, the situation immediately becomes clear:
Yes, but this is just one example. The point is that you cannot make assumptions about the re-use behavior of the memory manager, as I shown with the heaptrc.keepreleased example. GetFPCHeapStatus.CurrHeapUsed is not a tool thats useful for detecting memory leaks. It simply is not designed to do this.
It's a bit like you are trying to use a hammer to put a skrew in. Yes by hitting very hard the skrew will get into soft wood, but that doesn't mean it's good to do so.

When you want to measure memory leaks, use tools that are build for measuring memory leaks. Always use the tools fit for the job and don't try to use other things that where never intended to use this way

runewalsh

  • Full Member
  • ***
  • Posts: 121
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #13 on: July 24, 2023, 12:24:08 pm »
Warfley, GetFPCHeapStatus.CurrHeapUsed is correct for detecting leaks (e.g. in tests), it has nothing to do with reusing etc. Investigating by placing a breakpoint at GetMem shows that your code allocates not just TStringList but also a TEncoding (from inherited TStrings.Create) cached into a global and deallocated at the application exit.

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 12888
  • FPC developer.
Re: Memory leak with pair GetMem(P, 0) FreeMem(P, 0)
« Reply #14 on: July 24, 2023, 12:54:56 pm »
Could be related that freemem(p,0); is TP syntax which afaik didn't allow zero allocations

 

TinyPortal © 2005-2018