Recent

Author Topic: Heaptrace report missing details  (Read 14548 times)

kaapte

  • Newbie
  • Posts: 3
Heaptrace report missing details
« on: April 01, 2024, 09:32:40 pm »
Hi all,

I have a multithreaded FPC app running on linux. It's permanently running and over the course of the days and weeks, memory usage will suddenly jump up some 100MB or 200MB and never go back down. So I decided to try Heaptrace to find the cause of this.
Usually I use mORMot's fpcx64mm memory manager. The Heaptrace wiki says one should not use any other memory manager when using Heaptrace, so I removed it. However I noticed, memory usage is going up rapidly then, I was at 700MB after 30 minutes and it kept going up. Is that normal? Is Heaptrace collecting data and keep it in memory?

So to my actual issue: I set
Code: Pascal  [Select][+][-]
  1. export HEAPTRC="log=heap.trc"
to write the Heaptrace report into a file. That works, but it does not print any details about where in my source code the problems might be. See this report which shows a lot of unfreed memory blocks, but there is no more information. What might be the issue here?
I used these options for compiling: -dDEBUG -MDelphi -gl -gp -gw3 -gh

Thanks!

Quote
Heap dump by heaptrc unit of app
1268968142 memory blocks allocated : 154865380079/160289538184
1268845654 memory blocks freed     : 154839922875/160263834208
122488 unfreed memory blocks : 254

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Heaptrace report missing details
« Reply #1 on: April 02, 2024, 11:04:45 am »
-dDEBUG -MDelphi -gl -gp -gw3 -gh
Let's examine this.
1. -dDEBUG
How is your debug mode defined in fpc.cfg?
2. -MDelphi
should be -Mdelphi
3. -gl
That is ok.
4. -gp
you are not using stabs but dwarf. Makes no sense.
5. -gw3
This is already the default (in 3.2.2), so you can omit it.
6. -gh
the -g switch can be combined, so make it just one: -glh

But indeed, heaptrc gathers the information in memory and reports on program finished.
Since I did not see a whole of the heaptrc log:
It should report line numbers etc.
But what you have got is a big gaping hole that should be spotted by just reading the code.
(you have read the wiki, and understood that heaptrc should not be added by hand? just -glh will do )
First hang back and look what can cause such massive leak, if you can not find it report back. It should be visible by eye and heaptrc log should identify line numbers, except if you have many inc files. In that case it will be off by some lines.
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

kaapte

  • Newbie
  • Posts: 3
Re: Heaptrace report missing details
« Reply #2 on: April 02, 2024, 10:24:01 pm »
Thanks for your response.

1. -dDEBUG
How is your debug mode defined in fpc.cfg?
I don't have a fpc.cfg. So it must use some default mode, I'm not sure which options there are? I also added the -n parameter to make sure, there are no weird things inside a fpc.cfg somewhere.
2. -MDelphi
should be -Mdelphi
Alright, i changed that. However the delphi mode has been working already.
4. -gp
you are not using stabs but dwarf. Makes no sense.
Ok, I will remove that one and try again.

Since I did not see a whole of the heaptrc log:
It should report line numbers etc.
Yes you did see the whole heaptrace log. This is the main issue I'm facing now, that it only just writes these 4 lines into the report and no details like line numbers. Maybe it's just too much information (122488 unfreed memory blocks) and that causes an error. Or some other error preventing it to write the details. I will try a really short session to see if it will output details then.

But what you have got is a big gaping hole that should be spotted by just reading the code.
(you have read the wiki, and understood that heaptrc should not be added by hand? just -glh will do )
First hang back and look what can cause such massive leak, if you can not find it report back. It should be visible by eye and heaptrc log should identify line numbers, except if you have many inc files. In that case it will be off by some lines.

Yes, I noticed memory usage is continuously going up. That is only the case when using heaptrace and disabling mORMot's fpcx64mm memory manager (which is required according to the wiki).
This is a rather complex program and the memory leak usually happens within a single incident in varying time spans (like 1 week it's running at 350MB RAM and then suddenly jump to 450MB, then remain at 450MB for 2 days and then jump to 600MB for example). I'm not able to tell where the leak is, just by looking at the code, I'm afraid :)
Yes I read the wiki and did not add the heaptrace unit manually.

DavidL

  • New Member
  • *
  • Posts: 10
Re: Heaptrace report missing details
« Reply #3 on: April 02, 2024, 11:35:33 pm »

I've had to deal with a couple of large C++ programs that exhibited similar behavior.  In both cases it turned out that the OO kiddies didn't realize their "collection classes" chewed up increasingly larger chunks of memory when they hit their growth boundaries over extended run times.  Just a thought...

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Heaptrace report missing details
« Reply #4 on: April 03, 2024, 07:46:10 am »
Thanks for your response.

1. -dDEBUG
How is your debug mode defined in fpc.cfg?
I don't have a fpc.cfg. So it must use some default mode, I'm not sure which options there are? I also added the -n parameter to make sure, there are no weird things inside a fpc.cfg somewhere.
There are no defaults for the debug conditional. The debug settings always come from an fpc.cfg but it may have a different cfg name (although few people use that feature) Or the debug settings may be defined on the commandline of course, but not as -dDEBUG. Which will do nothing if there is no config....

Since you are on Linux, try to set up compilation for valgrind with -gv and use valgrind with a lot of verbosity. And/or use -vv which writes an fpcdebug.txt file with lots of additional info.
There should be an fpc.cfg in e.g. /etc/fpc.cfg if it is not under /usrl/bin/fpc/<version> or /usr/local/bin/fpc/<version>. The one in /etc/ is sticky and often overlooked.
I am almost sure that if you do cat /etc/fpc.cfg you will see a config file.....
« Last Edit: April 03, 2024, 08:09:14 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

kaapte

  • Newbie
  • Posts: 3
Re: Heaptrace report missing details
« Reply #5 on: April 03, 2024, 08:24:08 pm »
Oh I completely missed the one in /etc/fpc.cfg.
There it says:

Code: [Select]
#IFDEF DEBUG
  -gl
  -Crtoi
  #WRITE Compiling Debug Version
#ELSE
  # Strip debuginfo from the executable if not in debug mode
  -Xs
#ENDIF

I was trying to avoid using Valgrind because it's limiting the program to only using 1 thread/core. The program is highly multithreaded and I was afraid this would probably mean the issue might not even come up when it's only running 1 thread at a time.
However I will also give Valgrind a try.

Thanks for your help, guys. I will post any news here.

Thaddy

  • Hero Member
  • *****
  • Posts: 14376
  • Sensorship about opinions does not belong here.
Re: Heaptrace report missing details
« Reply #6 on: April 04, 2024, 09:03:09 am »
I was trying to avoid using Valgrind because it's limiting the program to only using 1 thread/core.
That is not true, Valgrind will happily target multiple threads.
With the provision that Valgrind will analyze them sequentially, that is true.
« Last Edit: April 04, 2024, 09:05:50 am by Thaddy »
Object Pascal programmers should get rid of their "component fetish" especially with the non-visuals.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5481
  • Compiler Developer
Re: Heaptrace report missing details
« Reply #7 on: April 07, 2024, 10:11:33 pm »
2. -MDelphi
should be -Mdelphi

This is irrelevant, because the mode(switch) name is case insensitive.

 

TinyPortal © 2005-2018