* * *

Author Topic: Is there anyway to apply filter to heap.trc memory leak list?  (Read 574 times)

Awesome Programmer

  • Sr. Member
  • ****
  • Posts: 308
  • Programming is FUN only when it works :)
    • Cool Technology
Is there anyway to apply filter to heap.trc memory leak list?
« on: December 05, 2017, 08:21:27 pm »
I have a feeling that my software has memory leak. So, I enabled heaptrc for my program and ran it. Then, I opened the heap.trc file and it has a LOTS AND LOTS of stuff in it. It must be over 100 pages long or something. And I hate to think that my software has that many memory leak. lol... So, is there a way to apply filter to heap trace? So, you only get relevant memory leak or real memory leak.

Thanks,

taazz

  • Hero Member
  • *****
  • Posts: 4485
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #1 on: December 05, 2017, 08:35:48 pm »
leakview http://wiki.lazarus.freepascal.org/leakview compile it and install it. IT makes things easier. Also when the leak count is big in my code usually everything merges in to a couple points. My max count so far was 4 leak corrections on a multi megabyte file that took around 1 minute and a half to write when exiting my application.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4416
    • wiki
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #2 on: December 05, 2017, 08:57:25 pm »
If you get huge lists, that may be because of...

Code: Pascal  [Select]
  1. TFoo = class
  2.   FA, FB: string;
  3.   FX: Array of String; // SEtLenght(10)
  4.   // ...
  5. end

leak one instance of TFoo, and you got  14 leaks. Leak 2 instances and you got 28 leaks.

1 instance consists of (assuming that none of the strings is empty)
- the object
- FA
- FB
- FX
- 10 strings in FX

In the heaptrc list the object may be listed last (not sure why, but I often saw it that way)

So start with the last trace in the log. (and use leakview)

Awesome Programmer

  • Sr. Member
  • ****
  • Posts: 308
  • Programming is FUN only when it works :)
    • Cool Technology
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #3 on: December 06, 2017, 04:38:27 pm »
Thank you for the leakview post. It is great. After I examined the heap.trc file, I noticed that all of the leaks mentioned in the file is during SHUTDOWN, which I personally don't care for. What I am more concerned with is memory leak during runtime. Is there away to capture runtime memory leak in heap.trc?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4416
    • wiki
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #4 on: December 06, 2017, 05:47:48 pm »
I noticed that all of the leaks mentioned in the file is during SHUTDOWN, which I personally don't care for. What I am more concerned with is memory leak during runtime. Is there away to capture runtime memory leak in heap.trc?

You mean:
1) they are reported when your app closes
2) they are caused inside Form1.Destroy (or unit finalization section)?

leakview always only reports during shutdown. Before that it does not know if the memory may still be freed by your app.
So a leak for heaptrc is defined as:
- Memory that was allocated at some time (while your app runs, including startup, shutdown, and all else)
AND
- Memory that was not freed before shutdown (never mind how long ago it should have been freed)


Code: Pascal  [Select]
  1.  a := TFoo.create;
  2.  a := TFoo.create; // here you leak the first TFoo, because you no longer have a reference to it, so you can not free it anymore.
  3.  
heaptrc does detect that leak only at shutdown. Not at the time described as the leak in the comment.

Afaik (not sure) valgrind can do better. But that is linux only.

Awesome Programmer

  • Sr. Member
  • ****
  • Posts: 308
  • Programming is FUN only when it works :)
    • Cool Technology
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #5 on: December 06, 2017, 07:48:55 pm »
oh I see...

You are right about FormDestroy. All of the memory leak that is listed in heaptrc is at SHUTDOWN.

However, all it says is "at line 733 in epmain.pas." you go there and it is of course FormDestroy. This is too vague. In this case, it could be anything. If your program contains 100 files, it would be like finding a needle in a haystack.

I guess I need to look for something else. :)

taazz

  • Hero Member
  • *****
  • Posts: 4485
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #6 on: December 06, 2017, 08:06:56 pm »
oh I see...

You are right about FormDestroy. All of the memory leak that is listed in heaptrc is at SHUTDOWN.

However, all it says is "at line 733 in epmain.pas." you go there and it is of course FormDestroy. This is too vague. In this case, it could be anything. If your program contains 100 files, it would be like finding a needle in a haystack.

I guess I need to look for something else. :)
No that is your start of the problem, do you have any controls created dynamically? Are they Components and if yes do they have proper ownership? after that look for dynamically created non components in that file do you have a destroy event to free them? IF not why?
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Awesome Programmer

  • Sr. Member
  • ****
  • Posts: 308
  • Programming is FUN only when it works :)
    • Cool Technology
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #7 on: December 08, 2017, 03:11:13 pm »
i see...

but are there any linux program like process viewer for windows that I can run on Linux OS that will enable me to see my software running live and all the resources it is using?

All the leaks leakview with heap.trc has shown me so far are nothing but leaks that I can indefinitely IGNORE. Yes, I do see in some of my code where I do create a variable or an object that can be released after the software is done using them, but it is not necessary. They go away when you shutdown your program anyhow. The leaks I am really interested in are the ones that are created repeatedly and not released during run time.

From what you are saying, then I have to let my program run for a while maybe even an hour and then see the result of the heap.trc file and see which leak is huge in size.

Thaddy

  • Hero Member
  • *****
  • Posts: 4780
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #8 on: December 08, 2017, 03:21:43 pm »
i see...

but are there any linux program like process viewer for windows that I can run on Linux OS that will enable me to see my software running live and all the resources it is using?
Yes, "top" simply type top in a terminal. https://www.lifewire.com/linux-top-command-2201163 and https://linux.die.net/man/1/top
« Last Edit: December 08, 2017, 03:27:33 pm by Thaddy »
"Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory, but a single counterexample is logically decisive."

molly

  • Hero Member
  • *****
  • Posts: 2039
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #9 on: December 08, 2017, 03:25:52 pm »
All the leaks leakview with heap.trc has shown me so far are nothing but leaks that I can indefinitely IGNORE. Yes, I do see in some of my code where I do create a variable or an object that can be released after the software is done using them, but it is not necessary. They go away when you shutdown your program anyhow. The leaks I am really interested in are the ones that are created repeatedly and not released during run time.
And with that you created a chicken and egg problem.

Had you developed from the start with heaptrace you could then'fix' any new reported leak right from the start. If you fix those "at program end" then heaptrace will show you those leaks that where created at runtime. fwiw: the ones you wish to ignore are also created at runtime (even though you think it is after your program ended). You have no idea what and how many code is running when you think your program has ended (that is actually quite a bit). All that code has to deal with your leaks that you deem not important to fix because as you say: they get fixed themselves. (which btw is a mistake as well, in case you are leaking host resources).

Thaddy

  • Hero Member
  • *****
  • Posts: 4780
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #10 on: December 08, 2017, 03:33:30 pm »
All the leaks leakview with heap.trc has shown me so far are nothing but leaks that I can indefinitely IGNORE.
If you do that under my watch you are fired. A memory leak in a compiled language is something you must never, ever ignore.
E.g. Windows recovers pretty well (but not totally) but other OS's need a restart to release such resources.
If you care to IGNORE that you are not an awesome programmer.

Same as molly explained: be as cocky as you like at your own peril. (I won't even add grumpy, it's not worth it)

Maybe you can enlighten us why you can indefinitely ignore such leaks < sardonic  >:D> ?
« Last Edit: December 08, 2017, 03:40:38 pm by Thaddy »
"Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory, but a single counterexample is logically decisive."

Awesome Programmer

  • Sr. Member
  • ****
  • Posts: 308
  • Programming is FUN only when it works :)
    • Cool Technology
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #11 on: December 08, 2017, 10:58:11 pm »
I am so sorry if I came off as being cocky.

I thought memory leak is when a program or a process keeps on hogging resources little at a time over long period of time. Eventually, that leading to system failure or program crash not when for instance...

procedure TAnimals.MakeCats;
var
myCat:TheCat;
begin
myCat := TheCat.Create;
end;

Even though myCat object is not released after being created, it obviously is not LEAKING memory because it only needs X number of memory when myCat object is created. Correct me if I am wrong. Well, I am working with a program that was written by someone else years ago and I believe the program has memory leak. The program is supposed to run 24/7/365 days forever and ever... but it fails or crashes after about a month or so. I am stepping through the code and correcting whatever leakview is showing me...

Thanks...

lainz

  • Hero Member
  • *****
  • Posts: 2087
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #12 on: December 08, 2017, 11:01:44 pm »
The code you provided IS a memory leak.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4416
    • wiki
Re: Is there anyway to apply filter to heap.trc memory leak list?
« Reply #13 on: December 08, 2017, 11:41:02 pm »
I thought memory leak is when a program or a process keeps on hogging resources little at a time over long period of time. Eventually, that leading to system failure or program crash not when for instance...

I would call any memory that is not explicitly returned a memory leak.
A once off leak is still a leak.
A "harmless" leak, is still a leak.
A leak "repaired" by your OS (free on exit) is still a leak.

If you allocate one instance of TFoo (and one only), and you need it throughout the entire lifetime of your app, you should still return it.

Even if a once off memory leak may not harm you. (That is, if the OS reclaims the mem on app exit).

--------------------
In any case: leaktrc (and maybe other tools too) can not tell the difference between a memory leak that is intended, and a leak that may run you out of memory.

That means, if you choose to ignore some of them, then it becomes harder to check for the others.

---------------------
As I said valgrind may have more options. I dont know if it has an ignore list or not. But even if it has, you have to maintain it. And as your app changes, the same leak may need a diff entry in such a list (as the leak "moves"). So even with an ignore list (if there is one) it will still be extra work to do the ignoring...


 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus