HeapTrc not showing line numbers

[TRon]

No mention of line 6. Starts from the method that called the leaking method, not the leak !

Yeah, I mentioned that some time ago as well in combination with Lazarus.

Common reaction was: you have to learn how to read the trace even if that leads to dark obscure places. There is practically no use for heaptrace on linux except for knowing that there is a leak and that it is somewhere in the neighbourhood of the displayed trace (but in practice it can be buried anywhere)

:shrugs:

[dbannon]

Yes TRon, well summed up.

I have confirmed, even with a March 2024 FPC-Fixes, no line numbers. (Intel based Monterey).

To be quite clear, on Mac, heaptrc is useful to tell you you have a leak, but does not tell you where.  Lets not go off chasing this particular wild goose again.

This is almost certainly a bug in the FPC / heaptce tool chain, I totally reject Thaddy's suggestion that we have discovered a flaw in Unix ! (I think he must have been joking...)

JdeHaan, if you can get your code to compile on Linux (or even Windows) debug there. VirtualBox perhaps ? Otherwise, it might be a case of gradually commenting out sections of code until the leak goes away (or reduces), messy indeed. My 25K line project does not have any leaks, obviously I don't use all of LCL but some of it, so as a starting point, assume the leaks are in your own code. Sorry

I do all my development on Linux, its so much easy to debug there. (Join us, do not be afraid....)

Davo

[JdeHaan]

@Davo, I used your example, with really the latest trunk (1 minute ago) for MacOs. I get this result:

Code: Text  [Select][+][-]

1. 1 memory blocks allocated : 16/16
2. 0 memory blocks freed     : 0/0
3. 1 unfreed memory blocks : 16
4. True heap size : 360448 (96 used in System startup)
5. True free heap : 360128
6. Should be : 360144
7. Call trace for block $0000000108B24100 size 16
8.   $000000010890D4D9
9.   $000000010890D4E9
10.   $000000010890D52E
11.   $0000000108930B9C
12.   $00007FF8178E1366
13.  

So, just 1 reporting trace, instead of your 3 traces...

I get your point on Linux, but I'm using Mac already for many years, and not ready to move to a different platform...
However, indeed let's close the topic for now.

[jamie]

has anyone tried using an external debug file?

I am sure the heaptrc must use something to reference by, maybe the MAC os is somehow not working the same way with the debug file attached to the bin file.

[dbannon]

From memory, Jamie, back in 2017, I was trying to track down a lean in Carbon's System Tray, then I was using gdb and the issue was that the compiler just did not put that information in there.  But today, of course, its lldb, Cocoa etc, so maybe ? Probably worth trying but I don't know how to do it !

Davo

[jamie]

I don't know a lot of MAC OS but I believe items are not bound together like they are in other OS's, for example Windows.

 If you go into the debugging options in your project from lazarus, you can checkmark the option to use external file.

do a build all..

[JdeHaan]

That's it! Thanks Jamie!

Result of the example:

Code: Pascal  [Select][+][-]

1. program memleak;
2.  
3. {$mode objfpc}
4.  
5. var a:TObject;
6.  
7. procedure DoIt3;
8. begin
9.   a:=TObject.Create;
10. end;
11.  
12.  
13. procedure DoIt2;
14. begin
15.   DoIt3;
16. end;
17.  
18. procedure DoIt;
19. begin
20.     DoIt2;
21. end;
22.  
23. begin
24.     DoIt;
25. end.
26.  

is now:

Code: Text  [Select][+][-]

1. 1 memory blocks allocated : 16/16
2. 0 memory blocks freed     : 0/0
3. 1 unfreed memory blocks : 16
4. True heap size : 360448 (96 used in System startup)
5. True free heap : 360128
6. Should be : 360144
7. Call trace for block $000000010A2FE100 size 16
8.   $000000010A15E909 line 15 of memleak.lpr
9.   $000000010A15E919 line 20 of memleak.lpr
10.   $000000010A15E95E line 24 of memleak.lpr
11.   $000000010A16736C
12.   $00007FF8178E1366
13.  

[TRon]

That's it! Thanks Jamie!

Really ?

Are you able to sum up what settings/circumstances you used to get that working because this (macos specific) topic seem to be a re-occurring one (and one that should be mentioned in the wiki).

TIA

[JdeHaan]

Sure, I attached a screenshot of my debugging settings.

Compilation and linking:
- Optimization level 0 or 1 (2, 3, 4 don't provide line numbers)

Debugging:
- Display line numbers (-gl)
- Use external debug symbols file (-Xg)
- Use heaptrc unit (-gh)


On MacOs Sonoma 14.4.1, with FPC trunk

[dbannon]

Wow, thats something of a breakthrough !

I am away from home for a couple of weeks, don't have access to my Mac. So cannot test and write something up on the Macos wiki page. Anyone else ?

Seriously, thanks JdeHaan and Jamie !

Davo

[TRon]

In order to keep things together I paste here verbatim the findings from mac user Wilko500 who gracefully accepted the challenge to try the suggested solution in this thread as well.

Because this thread has at least a relevant title this can hopefully become a gathering place for information exchange before becoming more definitive and be able to put this into the wiki.

Thank you to all participants in past, present and future and in case you are a mac user, please try this as well and share any findings. The more details, the better 

So the simple answer to "can I get heaptrc dump with line numbers" is yes and no. It appears to depend on what versions of FPC and Lazarus are being used.  My project development environment is Laz/Fpc =34/322 and that does NOT produce line numbers.  I also have a test environment of Laz/FPC = 399/331 that DOES produce line numbers.   Both were installed using FPCUpDeluxe (about screens attached.

The setting used are also included in pics and interestingly I already had those setting enabled in my development environment but no line numbers as the "wrong" versions used.

The post you referred me to spoke of having to use the trunk version(s).  I tried to install the trunks of both Laz & FPC using FPCUpDeluxe but found that anchor docking although installed did not do any docking so that test was abandoned so I cannot say whether the current trunk would show line numbers or not. 

I do find it surprising that this subject has been spoken of many times and that there IS a solution but it remains something of a mystery as to what exactly must be installed to make it happen. So my best guess at present is to install Laz/FPC = 399/331 using FPCUpDeluxe and then you should get line numbers in your heaptrc dump. Naturally if you have the necessary skills to download source and compile then there may be other options.  At present I do not have those skills.

Now, the subject of my original post about memory leaks and program corruption I have no answer, YET.  The dumps with line numbers are very long and I have had just a very quick look.  The results are interesting to say the least.  I have easily identified that two classes I created are not freed.  Ok, and easy fix and a learning point for me.  More interesting is a repeating set of references to a routine that opens an SQLite database does a lookup and closes the database (perhaps 1000's of times).   For this I thought that having opened a database then closing it would be enough.  Apparently not.

In addition there are many unfreed blocks that do not refer to any of my code.  These I do not understand at all.  Next I shall review my code in the light of the line numbered dumps to see if I can reduce the magnitude of the problem.  Maybe some more posts will follow as I try to make sense of it.

A huge thank you to all who have commented on this post. If I can assist with further tests (on Mac stuff) then yes, I'm up for that. I would be very pleased to contribute to an amazing product.

[carl_caulkett]

In order to keep things together I paste here verbatim the findings from mac user Wilko500 who gracefully accepted the challenge to try the suggested solution in this thread as well.

Because this thread has at least a relevant title this can hopefully become a gathering place for information exchange before becoming more definitive and be able to put this into the wiki.

Thank you to all participants in past, present and future and in case you are a mac user, please try this as well and share any findings. The more details, the better 

So the simple answer to "can I get heaptrc dump with line numbers" is yes and no. It appears to depend on what versions of FPC and Lazarus are being used.  My project development environment is Laz/Fpc =34/322 and that does NOT produce line numbers.  I also have a test environment of Laz/FPC = 399/331 that DOES produce line numbers.   Both were installed using FPCUpDeluxe (about screens attached.

The setting used are also included in pics and interestingly I already had those setting enabled in my development environment but no line numbers as the "wrong" versions used.

The post you referred me to spoke of having to use the trunk version(s).  I tried to install the trunks of both Laz & FPC using FPCUpDeluxe but found that anchor docking although installed did not do any docking so that test was abandoned so I cannot say whether the current trunk would show line numbers or not. 

I do find it surprising that this subject has been spoken of many times and that there IS a solution but it remains something of a mystery as to what exactly must be installed to make it happen. So my best guess at present is to install Laz/FPC = 399/331 using FPCUpDeluxe and then you should get line numbers in your heaptrc dump. Naturally if you have the necessary skills to download source and compile then there may be other options.  At present I do not have those skills.

Now, the subject of my original post about memory leaks and program corruption I have no answer, YET.  The dumps with line numbers are very long and I have had just a very quick look.  The results are interesting to say the least.  I have easily identified that two classes I created are not freed.  Ok, and easy fix and a learning point for me.  More interesting is a repeating set of references to a routine that opens an SQLite database does a lookup and closes the database (perhaps 1000's of times).   For this I thought that having opened a database then closing it would be enough.  Apparently not.

In addition there are many unfreed blocks that do not refer to any of my code.  These I do not understand at all.  Next I shall review my code in the light of the line numbered dumps to see if I can reduce the magnitude of the problem.  Maybe some more posts will follow as I try to make sense of it.

A huge thank you to all who have commented on this post. If I can assist with further tests (on Mac stuff) then yes, I'm up for that. I would be very pleased to contribute to an amazing product.

* Mac Mini M1
* macOS 14.6.1
* Lazarus 3.99
* FPC 3.3.1

I've used FpcupDeluxe to update to Laz 3.99 + FPC 3.3.1 as described by Wilko500, but I still cannot see any line number information. This is the type of thing I see...

Code: Text  [Select][+][-]

1. /Users/carlcaulkett/Code/FPC/OsmosePresets/OsmosePresets.app/Contents/MacOS/OsmosePresets
2. Heap dump by heaptrc unit of "/Users/carlcaulkett/Code/FPC/OsmosePresets/OsmosePresets"
3. 27500 memory blocks allocated : 30395928/30456696
4. 27448 memory blocks freed     : 30392093/30452824
5. 52 unfreed memory blocks : 3835
6. True heap size : 1900544 (128 used in System startup)
7. True free heap : 1884512
8. Should be : 1886560
9. Call trace for block $000000010F017500 size 152
10.   $000000010008EDED
11.   $000000010008E76A
12.   $0000000100078811
13.   $000000010007781F
14.   $0000000100014099
15.   $000000010022FFED
16.   $0000000100079A5D
17.   $000000010023B806
18.   $000000010022DB5C
19.   $000000010022DA9C
20.   $000000010007D862
21.   $000000010022BF0A
22.   $000000010024620D
23.   $0000000100242C19
24.   $0000000100077328
25.   $000000010007C312
26. Call trace for block $000000010F039E40 size 56
27.   $000000010008EDED
28.   $000000010008E76A
29.   $0000000100078811
30.   $000000010007781F
31.   $0000000100014099
32.   $000000010022FFED
33.   $0000000100079A5D
34.   $000000010023B806
35.   $000000010022DB5C
36.   $000000010022DA9C
37.   $000000010007D862
38.   $000000010022BF0A
39.   $000000010024620D
40.   $0000000100242C19
41.   $0000000100077328
42.   $000000010007C312
43.  

Does the fact that I am on a Mac make any difference as to whether I should be able to see the line numbers or not?

[Zvoni]

* Mac Mini M1
* macOS 14.6.1
* Lazarus 3.99
* FPC 3.3.1

I've used FpcupDeluxe to update to Laz 3.99 + FPC 3.3.1 as described by Wilko500, but I still cannot see any line number information. This is the type of thing I see...

Code: Text  [Select][+][-]

1. /Users/carlcaulkett/Code/FPC/OsmosePresets/OsmosePresets.app/Contents/MacOS/OsmosePresets
2. Heap dump by heaptrc unit of "/Users/carlcaulkett/Code/FPC/OsmosePresets/OsmosePresets"
3. 27500 memory blocks allocated : 30395928/30456696
4. 27448 memory blocks freed     : 30392093/30452824
5. 52 unfreed memory blocks : 3835
6. True heap size : 1900544 (128 used in System startup)
7. True free heap : 1884512
8. Should be : 1886560
9. Call trace for block $000000010F017500 size 152
10.   $000000010008EDED
11.   $000000010008E76A
12.   $0000000100078811
13.   $000000010007781F
14.   $0000000100014099
15.   $000000010022FFED
16.   $0000000100079A5D
17.   $000000010023B806
18.   $000000010022DB5C
19.   $000000010022DA9C
20.   $000000010007D862
21.   $000000010022BF0A
22.   $000000010024620D
23.   $0000000100242C19
24.   $0000000100077328
25.   $000000010007C312
26. Call trace for block $000000010F039E40 size 56
27.   $000000010008EDED
28.   $000000010008E76A
29.   $0000000100078811
30.   $000000010007781F
31.   $0000000100014099
32.   $000000010022FFED
33.   $0000000100079A5D
34.   $000000010023B806
35.   $000000010022DB5C
36.   $000000010022DA9C
37.   $000000010007D862
38.   $000000010022BF0A
39.   $000000010024620D
40.   $0000000100242C19
41.   $0000000100077328
42.   $000000010007C312
43.  

Does the fact that I am on a Mac make any difference as to whether I should be able to see the line numbers or not?

And did you set your debugging-options as shown in Post #23?
https://forum.lazarus.freepascal.org/index.php/topic,67035.msg515931.html#msg515931

[carl_caulkett]

And did you set your debugging-options as shown in Post #23?
https://forum.lazarus.freepascal.org/index.php/topic,67035.msg515931.html#msg515931

Thanks for the reply! As you were typing that, I had literally just copied the options from JdeHaan's post, and it worked 😃

[carl_caulkett]

Sure, I attached a screenshot of my debugging settings.

Thanks for posting this, I now have working line numbers in my `heaptrc` 😃