Lazarus

Free Pascal => General => Topic started by: hinst on April 20, 2014, 01:35:49 pm

Title: heaptrc output meaning?
Post by: hinst on April 20, 2014, 01:35:49 pm
What do these numbers mean?

Heap dump by heaptrc unit
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
2 unfreed memory blocks : 16424
True heap size : 2359296
True free heap : 2359296
Should be : 2342616
Title: Re: heaptrc output meaning?
Post by: jwdietrich on April 20, 2014, 02:28:04 pm
The meaning of the output is that your application has at least one memory leak. 25513 memory blocks were allocated, but only 25511 blocks have been freed after your program exits. You should check your code to find out which block was allocated but not released.
Title: Re: heaptrc output meaning?
Post by: hinst on April 20, 2014, 02:36:25 pm
And what about these numbers?
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
I often stare at them but do not understand what they mean. Is it 51 Mega Bytes? Where? When? Why?
Title: Re: heaptrc output meaning?
Post by: Leledumbo on April 20, 2014, 02:54:24 pm
From the source:
Code: [Select]
procedure dumpheap;
var
  pp : pheap_mem_info;
  i : ptrint;
  ExpectedHeapFree : ptruint;
  status : TFPCHeapStatus;
  ptext : ^text;
  loc_info: pheap_info;
begin
  loc_info:=@heap_info;
  if useownfile then
    ptext:=@ownfile
  else
    ptext:=@stderr;
  pp:=loc_info^.heap_mem_root;
  Writeln(ptext^,'Heap dump by heaptrc unit');
  Writeln(ptext^,loc_info^.getmem_cnt, ' memory blocks allocated : ',
    loc_info^.getmem_size,'/',loc_info^.getmem8_size);
  Writeln(ptext^,loc_info^.freemem_cnt,' memory blocks freed     : ',
    loc_info^.freemem_size,'/',loc_info^.freemem8_size);
  Writeln(ptext^,loc_info^.getmem_cnt-loc_info^.freemem_cnt,
    ' unfreed memory blocks : ',loc_info^.getmem_size-loc_info^.freemem_size);
...
Can you guess? ;)
Title: Re: heaptrc output meaning?
Post by: hinst on April 20, 2014, 02:57:25 pm
So it shows how much memory application allocated, but it does not mean that total memory usage reached this number at any specific point of time?
Title: Re: heaptrc output meaning?
Post by: Martin_fr on April 20, 2014, 04:56:41 pm
And what about these numbers?
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
I often stare at them but do not understand what they mean. Is it 51 Mega Bytes? Where? When? Why?

AFAIK (but not 100% sure)
blocks allocated is how often a bock of mem was allocated.
   Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10);
would be 4 allocs size 40, and 4 frees size 40, yet they probably used the same memory.


Title: Re: heaptrc output meaning?
Post by: Leledumbo on April 20, 2014, 05:39:52 pm
So it shows how much memory application allocated, but it does not mean that total memory usage reached this number at any specific point of time?
Sorry, I don't understand how you differ "memory application allocated" and "total memory usage reached" since both IMO are the same.
Title: Re: heaptrc output meaning?
Post by: karaba on April 20, 2014, 06:05:11 pm
Sorry, I don't understand how you differ "memory application allocated" and "total memory usage reached" since both IMO are the same.

I assume he means that the memory manager can allocate more memory than the application has requested as a pre-fetch optimization, that means that the code has allocated 90M but the manager has already allocated 100M for future use.
Title: Re: heaptrc output meaning?
Post by: hinst on April 20, 2014, 06:13:18 pm
That is because I don't know what is better term to call these numbers

   Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10);
would be 4 allocs size 40, and 4 frees size 40, yet they probably used the same memory.

Here peak is 10, while total is 40.

So I believe that hepatrc shows here total, not peak
  25513 memory blocks allocated : 51892490/51972360
  25511 memory blocks freed     : 51876066/51955936

I wonder what would happen if this internal memory counter overflows;
If I run a program like this
Code: [Select]
while True do
  FreeMem(GetMem(1000));
it should overflow sooner or later
the question is: when exactly? maybe it will take centuries
Title: Re: heaptrc output meaning?
Post by: engkin on April 20, 2014, 06:57:11 pm
25513 memory blocks allocated : 51892490/51972360
The first number is the sum of memory requests in bytes.
The second number is the previous number rounded up to the nearest 8 bytes boundary, so always dividable by eight (bytes on 64 bit architecture)
Title: Re: heaptrc output meaning?
Post by: engkin on April 20, 2014, 07:51:41 pm
I wonder what would happen if this internal memory counter overflows;
If I run a program like this
Code: [Select]
while True do
  FreeMem(GetMem(1000));
it should overflow sooner or later
the question is: when exactly? maybe it will take centuries
This number is of type PtrUInt:
On 32 bit applications it is 4 bytes so it overflows when it reaches 4,294,967,295. Which is the 4 GB limit of memory for one application in that architecture (32 address lines).
After going 4,294,967 times through that loop.

On 64 bit applications it is 8 bytes so it overflows when it reaches 18,446,744,073,709,551,615.
After going 18,446,744,073,709,551 times through that loop.  :P
Title: Re: heaptrc output meaning?
Post by: Leledumbo on April 21, 2014, 01:46:02 am
This number is of type PtrUInt:
On 32 bit applications it is 4 bytes so it overflows when it reaches 4,294,967,295. Which is the 4 GB limit of memory for one application in that architecture (32 address lines).
After going 4,294,967 times through that loop.

On 64 bit applications it is 8 bytes so it overflows when it reaches 18,446,744,073,709,551,615.
After going 18,446,744,073,709,551 times through that loop.  :P
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Title: Re: heaptrc output meaning?
Post by: engkin on April 21, 2014, 03:36:47 am
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Not really, the figures are added based on requests only:
Code: [Select]
Function TraceGetMem(size:ptruint):pointer;
..
begin
..
  inc(loc_info^.getmem_size,size);
  inc(loc_info^.getmem8_size,(size+7) and not 7);
..
  p:=SysGetMem(allocsize);
  if (p=nil) then
    begin
      TraceGetMem:=nil;
      exit;
    end;
Title: Re: heaptrc output meaning?
Post by: hinst on April 21, 2014, 10:29:03 am
Indeed. Perhaps a better solution would be to use 64-bit unsigned integer, not PtrUInt for
getmem_size, getmem8_size, freemem_size and freemem8_size
Otherwise it could overflow in 32-bit program quickly
Title: Re: heaptrc output meaning?
Post by: Leledumbo on April 21, 2014, 04:51:58 pm
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Not really, the figures are added based on requests only:
I mean, there would already be an out of memory error from the OS before 4 GB and 64 GB is reached if the system itself doesn't have physical + virtual memory of that size.
Title: Re: heaptrc output meaning?
Post by: taazz on April 21, 2014, 08:26:11 pm
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Not really, the figures are added based on requests only:
I mean, there would already be an out of memory error from the OS before 4 GB and 64 GB is reached if the system itself doesn't have physical + virtual memory of that size.

The number sums up all the requests that where made for memory regardless if the memory is no longer allocated ee was disposed and used by the next allocation, since it never goes down on deallocation then if an application isvery aggresive on allocation/deallocation it is possible to overflow the counter and the application still use only 4MB of memory.