Recent

Author Topic: heaptrc output meaning?  (Read 10794 times)

hinst

  • Sr. Member
  • ****
  • Posts: 303
heaptrc output meaning?
« on: April 20, 2014, 01:35:49 pm »
What do these numbers mean?

Heap dump by heaptrc unit
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
2 unfreed memory blocks : 16424
True heap size : 2359296
True free heap : 2359296
Should be : 2342616
Too late to escape fate

jwdietrich

  • Hero Member
  • *****
  • Posts: 1036
    • formatio reticularis
Re: heaptrc output meaning?
« Reply #1 on: April 20, 2014, 02:28:04 pm »
The meaning of the output is that your application has at least one memory leak. 25513 memory blocks were allocated, but only 25511 blocks have been freed after your program exits. You should check your code to find out which block was allocated but not released.
function GetRandomNumber: integer; // xkcd.com
begin
  GetRandomNumber := 4; // chosen by fair dice roll. Guaranteed to be random.
end;

http://www.formatio-reticularis.de

Lazarus 2.0.6 | FPC 3.0.4 | PPC, Intel, ARM | macOS, Windows, Linux

hinst

  • Sr. Member
  • ****
  • Posts: 303
Re: heaptrc output meaning?
« Reply #2 on: April 20, 2014, 02:36:25 pm »
And what about these numbers?
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
I often stare at them but do not understand what they mean. Is it 51 Mega Bytes? Where? When? Why?
Too late to escape fate

Leledumbo

  • Hero Member
  • *****
  • Posts: 8111
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: heaptrc output meaning?
« Reply #3 on: April 20, 2014, 02:54:24 pm »
From the source:
Code: [Select]
procedure dumpheap;
var
  pp : pheap_mem_info;
  i : ptrint;
  ExpectedHeapFree : ptruint;
  status : TFPCHeapStatus;
  ptext : ^text;
  loc_info: pheap_info;
begin
  loc_info:=@heap_info;
  if useownfile then
    ptext:=@ownfile
  else
    ptext:=@stderr;
  pp:=loc_info^.heap_mem_root;
  Writeln(ptext^,'Heap dump by heaptrc unit');
  Writeln(ptext^,loc_info^.getmem_cnt, ' memory blocks allocated : ',
    loc_info^.getmem_size,'/',loc_info^.getmem8_size);
  Writeln(ptext^,loc_info^.freemem_cnt,' memory blocks freed     : ',
    loc_info^.freemem_size,'/',loc_info^.freemem8_size);
  Writeln(ptext^,loc_info^.getmem_cnt-loc_info^.freemem_cnt,
    ' unfreed memory blocks : ',loc_info^.getmem_size-loc_info^.freemem_size);
...
Can you guess? ;)

hinst

  • Sr. Member
  • ****
  • Posts: 303
Re: heaptrc output meaning?
« Reply #4 on: April 20, 2014, 02:57:25 pm »
So it shows how much memory application allocated, but it does not mean that total memory usage reached this number at any specific point of time?
Too late to escape fate

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
Re: heaptrc output meaning?
« Reply #5 on: April 20, 2014, 04:56:41 pm »
And what about these numbers?
25513 memory blocks allocated : 51892490/51972360
25511 memory blocks freed     : 51876066/51955936
I often stare at them but do not understand what they mean. Is it 51 Mega Bytes? Where? When? Why?

AFAIK (but not 100% sure)
blocks allocated is how often a bock of mem was allocated.
   Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10);
would be 4 allocs size 40, and 4 frees size 40, yet they probably used the same memory.



Leledumbo

  • Hero Member
  • *****
  • Posts: 8111
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: heaptrc output meaning?
« Reply #6 on: April 20, 2014, 05:39:52 pm »
So it shows how much memory application allocated, but it does not mean that total memory usage reached this number at any specific point of time?
Sorry, I don't understand how you differ "memory application allocated" and "total memory usage reached" since both IMO are the same.

karaba

  • New Member
  • *
  • Posts: 49
Re: heaptrc output meaning?
« Reply #7 on: April 20, 2014, 06:05:11 pm »
Sorry, I don't understand how you differ "memory application allocated" and "total memory usage reached" since both IMO are the same.

I assume he means that the memory manager can allocate more memory than the application has requested as a pre-fetch optimization, that means that the code has allocated 90M but the manager has already allocated 100M for future use.

hinst

  • Sr. Member
  • ****
  • Posts: 303
Re: heaptrc output meaning?
« Reply #8 on: April 20, 2014, 06:13:18 pm »
That is because I don't know what is better term to call these numbers

   Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10); Alloc(10); Free(10);
would be 4 allocs size 40, and 4 frees size 40, yet they probably used the same memory.

Here peak is 10, while total is 40.

So I believe that hepatrc shows here total, not peak
  25513 memory blocks allocated : 51892490/51972360
  25511 memory blocks freed     : 51876066/51955936

I wonder what would happen if this internal memory counter overflows;
If I run a program like this
Code: [Select]
while True do
  FreeMem(GetMem(1000));
it should overflow sooner or later
the question is: when exactly? maybe it will take centuries
Too late to escape fate

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: heaptrc output meaning?
« Reply #9 on: April 20, 2014, 06:57:11 pm »
25513 memory blocks allocated : 51892490/51972360
The first number is the sum of memory requests in bytes.
The second number is the previous number rounded up to the nearest 8 bytes boundary, so always dividable by eight (bytes on 64 bit architecture)

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: heaptrc output meaning?
« Reply #10 on: April 20, 2014, 07:51:41 pm »
I wonder what would happen if this internal memory counter overflows;
If I run a program like this
Code: [Select]
while True do
  FreeMem(GetMem(1000));
it should overflow sooner or later
the question is: when exactly? maybe it will take centuries
This number is of type PtrUInt:
On 32 bit applications it is 4 bytes so it overflows when it reaches 4,294,967,295. Which is the 4 GB limit of memory for one application in that architecture (32 address lines).
After going 4,294,967 times through that loop.

On 64 bit applications it is 8 bytes so it overflows when it reaches 18,446,744,073,709,551,615.
After going 18,446,744,073,709,551 times through that loop.  :P

Leledumbo

  • Hero Member
  • *****
  • Posts: 8111
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: heaptrc output meaning?
« Reply #11 on: April 21, 2014, 01:46:02 am »
This number is of type PtrUInt:
On 32 bit applications it is 4 bytes so it overflows when it reaches 4,294,967,295. Which is the 4 GB limit of memory for one application in that architecture (32 address lines).
After going 4,294,967 times through that loop.

On 64 bit applications it is 8 bytes so it overflows when it reaches 18,446,744,073,709,551,615.
After going 18,446,744,073,709,551 times through that loop.  :P
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: heaptrc output meaning?
« Reply #12 on: April 21, 2014, 03:36:47 am »
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Not really, the figures are added based on requests only:
Code: [Select]
Function TraceGetMem(size:ptruint):pointer;
..
begin
..
  inc(loc_info^.getmem_size,size);
  inc(loc_info^.getmem8_size,(size+7) and not 7);
..
  p:=SysGetMem(allocsize);
  if (p=nil) then
    begin
      TraceGetMem:=nil;
      exit;
    end;

hinst

  • Sr. Member
  • ****
  • Posts: 303
Re: heaptrc output meaning?
« Reply #13 on: April 21, 2014, 10:29:03 am »
Indeed. Perhaps a better solution would be to use 64-bit unsigned integer, not PtrUInt for
getmem_size, getmem8_size, freemem_size and freemem8_size
Otherwise it could overflow in 32-bit program quickly
Too late to escape fate

Leledumbo

  • Hero Member
  • *****
  • Posts: 8111
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: heaptrc output meaning?
« Reply #14 on: April 21, 2014, 04:51:58 pm »
With the assumption that the machine has 4 GB and 64 GB (respectively for 32-bit and 64-bit) of RAM [+ swap].
Not really, the figures are added based on requests only:
I mean, there would already be an out of memory error from the OS before 4 GB and 64 GB is reached if the system itself doesn't have physical + virtual memory of that size.