Forum > General

Finishing thread does not release Virtual memory

(1/5) > >>

jollytall:
I have an application that makes new threads all the time. They finish normally, no memory leak, etc..
Still the Virtual memory usage of the process is constantly growing. I read many articles and it seems to be the normal process, as they say, because calling the virtual memory deallocation is apparently an expensive process. So what happens, every time when a new thread is launched it reserves memory in the virtual space and when it terminates it releases all physical space, but not the virtual. The next process could maybe use the "abandoned" virtual memory (it is the same structure thread again), but instead it reserves a new virtual address space.
Since on x64 the space is large enough, it is not a frequent problem to deal with, but it is alarming to see that the virtual memory usage goes up fast and I also see some small increase in physical memory usage (maybe for the allocation table).
I would like to somehow release the virtual memory, even if it is a bit expensive.

Somewhere I read a topic about detaching the thread from the process and let it die alone and apparently in such a case the virtual memory reservation "dies" with the left alone thread. I am not sure it is possible, exists, but surely could not find a "DetachThread" procedure (C++ has it).

For information: I use Linux/Debian10 with Lazarus 2.0.12, FPC 3.2.0.

I know I could use a pool of threads to avoid this problem, but prefer not to rewrite the whole application if I can solve it in the current set-up.

marcov:
The classic cause is memory fragmentation.  A new block is allocated by the application, but it is larger than the empty spaces the heapmanager has, so it allocates it from the OS.

The solution is usually to pool large allocations manually, or break them up in smaller pieces.

Jonas Maebe:
If you set FreeOnTerminate of your TThread to true, it will run as a detached thread. Warning: in that case you cannot/must not use TThread.WaitFor on it (partly because in theory the thread could even end before you started waiting on it). You will need to implement your own synchronisation if you need similar functionality (e.g. by setting an event from the thread when it is about to finish).

jollytall:
@marcov,
I think it is unlikely. I call the same object type with the same create parameters many times. They should allocate the same memory, so it should fit in to an earlier one. Even if they are slightly different size (for whatever reason), it is unlikely that when I do it for the 100th time, none of the previous 99 is the same size or larger to be reused. I think that the system does not look for virtual memory blocks allocated but not used by the same process, once a new thread is created.

@Jonas
Thanks, it might be leading to the solution. Two problems:
I do not use TThread, but the one level lower BeginThread, so I would need to call whatever has to be called manually. It would not be a problem if I knew what to call. I also have my own Wait implementation, so that is not a problem either.
I check the TThread implementation and FFreeOnTermiante I found only once in ThreadProc, and it calls Free (i.e. Destroy), but in Destroy I did not find what to do. Can you help?

MarkMLl:

--- Quote from: jollytall on January 11, 2022, 11:58:55 am ---I would like to somehow release the virtual memory, even if it is a bit expensive.

--- End quote ---

On Linux,  swapoff -a  should do it.

Yes, it's extreme. But it should have the effect of (a) forcing everything briefly into real memory and (b) in order to do this minimising the amount of demand-loaded executable in memory. You probably want to run  swapon -a  shortly afterwards.

Apart from that, listen to Jonas et al.

MarkMLl

Navigation

[0] Message Index

[#] Next page

Go to full version