[SOLVED] Help with memory leaks

[cpicanco]

Hi everyone,

I am having a difficult time trying to solve some memory leaks (attached).

The first thing I did was to check -gh, -gl and all "Checks and Assertions" options in Lazarus project options. Also, I have Dwarf3 as debugger selected.

When I started, I noticed some easy to fix leaks, related to missing instances in destroy. Also, I fixed some missing "free" calls at program end.

But now, I am receiving some leak messages from Generics.Collections TFPList.Add, TFPObjectList.Add and also from SetLength, what does not make any sense for me right now.

A bunch of the messages in the attached log have traces starting at an EndTrial event, from SDLEvents singleton, an event handler class. For testing purposes, I tried calling the loop without passing through the event handler and memory leaks persisted. Any help will be much appreciated.

 

[Fibonacci]

58 unfreed memory blocks

If your app runs longer, are there more unfreed mem blocks? If not, I wouldnt care.

[Martin_fr]

The best way is to start at the bottom, because often the order turns out, that some object is holding several leaks (via its fields) and the object is further down in the list. In those cases solving the one at the bottom, solves some of the others.


On top of your list you got stuff like

Code: Text  [Select][+][-]

1.   $000000010000D652  fpc_dynarray_setlength,  line 216 of ../inc/dynarr.inc
2. Call trace for block $00000000065C5B30 size 26
3.   $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
4.   $00000001000088B3  NEWANSISTRING,  line 123 of ../inc/astrings.inc
5.   $0000000100009D9A  FPC_TRUELY_ANSISTR_UNIQUE,  line 878 of ../inc/astrings.inc
6. Call trace for block $0000000006585B80 size 64
7.   $000000010000D7F9  fpc_dynarray_setlength,  line 274 of ../inc/dynarr.inc
8. Call trace for block $00000000065C5E30 size 25
9.   $00000001000088B3  NEWANSISTRING,  line 123 of ../inc/astrings.inc
10.   $0000000100009C3B  fpc_ansistr_setlength,  line 793 of ../inc/astrings.inc
11.   $0000000100008D8F  fpc_ansistr_concat,  line 292 of ../inc/astrings.inc
12.  

Those a refcounted strings and arrays.

If you have somewhere "a := GetSomeArray;" and then later "b:=a" then b may be holding the memory and causing the leak. But it was allocated using "a" and the trace points there. That is one of the hardest leaks to trace.

Then you have stuff like

Code: Text  [Select][+][-]

1.   $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
2.   $00000001001CA4F5  Create,  line 1481 of generics.collections.pas
3.  

Those are leaked objects (most likely missing free/destroy). In some cases the traces are to short (they end in sdl units, and likely your code triggering them is further out.
Getting longer traces afaik needs to make changes to the heaptrc unit, and rebuild fpc.
But if you can run on Linux, "valgrind" can get you longer traces.

Further down you have

Code: Text  [Select][+][-]

1. Call trace for block $000000000165D6F0 size 192
2.   $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
3.   $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
4.   $00000001001E978C  Create,  line 287 of src/sdl.app.audio.recorder.devices.pas
5.   $00000001001E8549  Create,  line 82 of src/sdl.app.audio.recorder.pas
6.   $00000001001D586B  Create,  line 128 of src/sdl.app.audio.pas
7.   $00000001001B8764  Create,  line 209 of src/sdl.app.pas
8.   $000000010004407D  ButtonRunSessionClick,  line 83 of src/forms.main.pas
9.   $000000010013FBD1  CLICK,  line 2974 of include/control.inc
10.  

That trace goes to your code. Looks like missing Free/Destroy.

If you are lucky some of the other go away if you fix this.

[cpicanco]

Thank you Martin_fr. I will keep trying based your tips. For now, I solved one leak (and a bug too):

Call trace for block $000000000165D890 size 192
  $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
  $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
  $00000001001EA27C  Create,  line 406 of src/sdl.app.audio.recorder.devices.pas
  $00000001001E859A  Create,  line 84 of src/sdl.app.audio.recorder.pas
  $00000001001D586B  Create,  line 128 of src/sdl.app.audio.pas
  $00000001001B8764  Create,  line 209 of src/sdl.app.pas
  $000000010004407D  ButtonRunSessionClick,  line 83 of src/forms.main.pas
  $000000010013FBD1  CLICK,  line 2974 of include/control.inc
  $000000010015CDAF  CLICK,  line 55 of include/buttoncontrol.inc
  $000000010015D466  CLICK,  line 169 of include/buttons.inc
  $000000010015CCAE  WMDEFAULTCLICKED,  line 21 of include/buttoncontrol.inc
  $000000010000E6A2  DISPATCH,  line 694 of ../inc/objpas.inc
  $000000010013E9E9  WNDPROC,  line 2304 of include/control.inc
  $00000001001334ED  WNDPROC,  line 5420 of include/wincontrol.inc
  $00000001001AAD45  DELIVERMESSAGE,  line 114 of lclmessageglue.pas
  $0000000100114A71  DOWINDOWPROC,  line 2616 of win32/win32callback.inc
Call trace for block $000000000165D6F0 size 192
  $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
  $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
  $00000001001E978C  Create,  line 287 of src/sdl.app.audio.recorder.devices.pas
  $00000001001E8549  Create,  line 82 of src/sdl.app.audio.recorder.pas
  $00000001001D586B  Create,  line 128 of src/sdl.app.audio.pas
  $00000001001B8764  Create,  line 209 of src/sdl.app.pas
  $000000010004407D  ButtonRunSessionClick,  line 83 of src/forms.main.pas
  $000000010013FBD1  CLICK,  line 2974 of include/control.inc
  $000000010015CDAF  CLICK,  line 55 of include/buttoncontrol.inc
  $000000010015D466  CLICK,  line 169 of include/buttons.inc
  $000000010015CCAE  WMDEFAULTCLICKED,  line 21 of include/buttoncontrol.inc
  $000000010000E6A2  DISPATCH,  line 694 of ../inc/objpas.inc
  $000000010013E9E9  WNDPROC,  line 2304 of include/control.inc
  $00000001001334ED  WNDPROC,  line 5420 of include/wincontrol.inc
  $00000001001AAD45  DELIVERMESSAGE,  line 114 of lclmessageglue.pas
  $0000000100114A71  DOWINDOWPROC,  line 2616 of win32/win32callback.inc

These two leaks were two unfreed threads. The sdl.app.audio.recorder.devices.pas unit have a TAudioDevice(TThread) and the program have two audio devices (one for recording, one for playblack). So, the threads were hanging infinitely because their execute method waits forever for a RTLEvent. I thought that calling

RTLEventDestroy(Event);

before terminating the thread would be enough for releasing the event. I guess I was wrong. Calling

RTLEventSetEvent(Event);
RTLEventDestroy(Event);

before terminating solved these leaks.

[Martin_fr]

Well, ok.

So given that you likely need longer stack traces for the other leaks: Is it possible to build and run on Linux?

If so, compile witch   -gv
and WITHOUT -gh (no heaptrc)

then run
   valgrind --tool=memcheck   --num-callers=50  yourapp

[cpicanco]

Thank you Martin_fr, but right now I can't compile on linux.

1) I managed to fix some more leaks by moving the ".Free" related calls to the same unit where they first happens.

1.1) In one particular case (in the session.configurationfile.pas unit), I refactor it. Now, instead of creating a different TStringList for each trial loop and sending it to be freed by the TrialEnd call from outside, I am creating a single TStringList instance, calling TStringList.Clear at the appropriate place each loop.

1.2) In another particular case (in sdl.app.stimulus.factory), a factory created TStimulus instances and they were freed outside. Now a "clear" method is called from outside at TrialEnd and all TStimulus created are freed by the factory.

[Martin_fr]

Well you can build your own fpc (e.g. with FpcUpDeluxe).

Either patch unit heaptrc...

Or if you compile with -dFPC_HEAPTRC_EXTRA then you get more lines in the trace (and some other xtra checks)


Mind that strings (and dyn array) are only semi-thread save. If you pass them between threads you may need to have critical sections.
Some parts of the ref counting are thread save. But only some.

However most likely strings and dyn array leaks are created by the objects that are leaked. And fixing those will solve them.


there is still at least one leak with a trace back to your code

Code: Text  [Select][+][-]

1. Call trace for block $00000000015590B0 size 328
2.   $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
3.   $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
4.   $00000001001F3F8C  Create,  line 48 of src/sdl.app.graphics.button.pas
5.   $00000001001F2AF0  Create,  line 242 of src/sdl.app.stimuli.mts.pas
6.   $00000001001F1340  Create,  line 49 of src/sdl.app.trials.mts.pas
7.   $00000001001BAF3A  Play,  line 96 of src/sdl.app.trials.factory.pas
8.   $00000001001BB6D5  Play,  line 99 of src/session.blocks.pas
9.   $00000001001BB43D  InterTrialEventsEnd,  line 61 of src/session.blocks.pas
10.   $00000001001BBD3C  InterTrialEnd,  line 188 of src/session.intertrial.pas
11.   $00000001001BBB8D  TrialEnd,  line 114 of src/session.intertrial.pas
12.   $00000001001CD1C7  DoOnEndTrial,  line 103 of src/sdl.app.events.custom.pas
13.   $00000001001CD0F8  UserEvent,  line 123 of src/sdl.app.events.custom.pas
14.   $00000001001CCAC7  HandleEvent,  line 314 of src/sdl.app.events.abstract.pas
15.   $00000001001CCC8D  HandlePending,  line 485 of src/sdl.app.events.abstract.pas
16.   $00000001001B8964  Run,  line 229 of src/sdl.app.pas
17.   $0000000100044232  ButtonRunSessionClick,  line 100 of src/forms.main.pas
18.  

[cpicanco]

there is still at least one leak with a trace back to your code

Code: Text  [Select][+][-]

1. Call trace for block $00000000015590B0 size 328
2.   $0000000100010CC2  GETMEM,  line 286 of ../inc/heap.inc
3.   $000000010000E296  NEWINSTANCE,  line 441 of ../inc/objpas.inc
4.   $00000001001F3F8C  Create,  line 48 of src/sdl.app.graphics.button.pas
5.   $00000001001F2AF0  Create,  line 242 of src/sdl.app.stimuli.mts.pas
6.   $00000001001F1340  Create,  line 49 of src/sdl.app.trials.mts.pas
7.   $00000001001BAF3A  Play,  line 96 of src/sdl.app.trials.factory.pas
8.   $00000001001BB6D5  Play,  line 99 of src/session.blocks.pas
9.   $00000001001BB43D  InterTrialEventsEnd,  line 61 of src/session.blocks.pas
10.   $00000001001BBD3C  InterTrialEnd,  line 188 of src/session.intertrial.pas
11.   $00000001001BBB8D  TrialEnd,  line 114 of src/session.intertrial.pas
12.   $00000001001CD1C7  DoOnEndTrial,  line 103 of src/sdl.app.events.custom.pas
13.   $00000001001CD0F8  UserEvent,  line 123 of src/sdl.app.events.custom.pas
14.   $00000001001CCAC7  HandleEvent,  line 314 of src/sdl.app.events.abstract.pas
15.   $00000001001CCC8D  HandlePending,  line 485 of src/sdl.app.events.abstract.pas
16.   $00000001001B8964  Run,  line 229 of src/sdl.app.pas
17.   $0000000100044232  ButtonRunSessionClick,  line 100 of src/forms.main.pas
18.  

Catching this one solved all the remaing leaks! Thanks, you are awesome!

First, I noticed that everything was ok with LButton (Free and Destroy) in sdl.app.stimuli.mts.pas:

Code: Pascal  [Select][+][-]

1. constructor TMTSStimuli.Create;
2. begin
3.   inherited Create;
4.   FButton := TButton.Create;
5.   FSamples := TCustomStimulusList.Create;
6.   FComparisons := TCustomStimulusList.Create;
7.   FMTSModality.Samples := ModalityNone;
8.   FMTSModality.Comparisons := ModalityNone;
9.  
10.   TStimulusFactory.Clear;
11. end;
12.  
13. destructor TMTSStimuli.Destroy;
14. begin
15.   TStimulusFactory.Clear;
16.   FButton.Free;
17.   FSamples.Free;
18.   FComparisons.Free;
19.   inherited Destroy;
20. end;
21.  

Then I moved up to and place some debug lines to see if some trial was not being released by the trial factory (sdl.app.trials.factory.pas) and found that the last trial was missing a "FreeCurrentTrial;".

And now heaptrc is full of beautiful zeros! 

C:\Users\Rafael\Documents\GitHub\stimulus-control-sdl2\experiment.exe
Heap dump by heaptrc unit of "C:\Users\Rafael\Documents\GitHub\stimulus-control-sdl2\experiment.exe"
658156 memory blocks allocated : 42949574/45093368
658156 memory blocks freed     : 42949574/45093368
0 unfreed memory blocks : 0
True heap size : 1507328 (608 used in System startup)
True free heap : 1506720