Threads and memory leak - is it just the OS(?)

[AlanTheBeast]

I have a command line program that issues threads on a regular basis.  (Most tasks are not threaded, but some do best de-coupled from the main program than other tasks as they take longer and are more I/O dense).  This program runs continuously and does various "maintenance" items for me and raises alarms in certain conditions.
(I won't bore you with that ).

I get the same results on an i7 iMac and M3 iMac. (fpc 3.2.0 and 3.2.2 respectively).

This program is of small size, compiled, runs at about 2MB of system memory.

But - in 4 hours or so it grows to 326MB.  Using HeapStatus is useless.  (Lots of string use, very little allocated memory (new(p)) - less than 1 kB).

So, I wrote a separate test program to "torture test" this (100 threads at a time - not doing anything related to I/O).   The program grows to 1GB of memory in about 14 hours.  The rate of growth slows over time.  (Threads are created when the number of threads is less than 103 as threads expire).

I've read various posts here about similar issues and conclude that the memory growth is the OS - probably setting up stack and heap space that is not soon deallocated as long as the program is running - though over time it seems to re-use what has been allocated (since the mem growth slows down).

Comments?

Suggestions to abate?  (I already set the stack size to 32KB on BeginThread).

I do have a memory creep check, so I can also simply restart the program when x mem is passed but that is inelegant.

I can post the test program if anyone wants to examine/tinker with it.
It is "Mac OS" ish, probably easy to adapt to Linux, a little more effort for Windows perhaps.

I suppose I could run it on a Rasp Pi in RTL only (no OS), but that stuff is in the cupboard presently.  I recall the Ultibo Rasp Pi fpc to be very well behaved wrt to memory despite threading and frequent and large use of the heap. 

(And many more thread related functions as well - that I wish were available on mainstream v. of fpc - if they are I haven't found them).

[LV]

I don't know how useful I will be. I use Windows on a desktop computer with 6 cores and 12 threads. Programs can potentially have 100 or more threads, but in real calculations, no more than 12 threads are used. A further increase in the number of threads reduces the efficiency to the level of the Trevithick steam locomotive.

I have fairly niche tasks related to parallel processing large amounts of data. The scheme for using threads is simple. The program is launched with the required number of threads, controlled by instances of the TEvent class, and threads "live" until the program is completed.

I have not noticed anything suspicious before, for example, when the program processed data larger than 4 GB for over 100 hours. Now I launched it in test mode with 6 threads within an hour. At the time of run, it was 94.5 MB, and after an hour, it was 93.6 MB.

GUI App, Lazarus 3.4, FPC 3.2.2, Windows 11.

[AlanTheBeast]

I don't know how useful I will be. I use Windows on a desktop computer with 6 cores and 12 threads. Programs can potentially have 100 or more threads, but in real calculations, no more than 12 threads are used. A further increase in the number of threads reduces the efficiency to the level of the Trevithick steam locomotive.

I have fairly niche tasks related to parallel processing large amounts of data. The scheme for using threads is simple. The program is launched with the required number of threads, controlled by instances of the TEvent class, and threads "live" until the program is completed.

I have not noticed anything suspicious before, for example, when the program processed data larger than 4 GB for over 100 hours. Now I launched it in test mode with 6 threads within an hour. At the time of run, it was 94.5 MB, and after an hour, it was 93.6 MB.

GUI App, Lazarus 3.4, FPC 3.2.2, Windows 11.

The main data point I take here is that Windows seems to manage this better than OS X.

As to thread counts, I agree that less is more for most cases, but there is no theoretical upper bound (of course the OS has to shuffle the threads around.  This has nothing to do with core count, btw - that just makes things faster).

In testing my test code I made a booboo resulting in over 6000 threads running.  Happily in monitor CMD-C took care of that.  I added a thread to get thread counts and kill the test program if over 550.  In the actual program of interest the limit is 21.

OS X does have a thread count upper bound per logged in account - and the user can modify this limit.  It is many thousand.  (for the OS and all apps combined).  I'm too lazy to find the command that tells one what the limit is set at.  OS' do not care (should not) how many threads there are.  It's just another element in a linked list with a bunch of parameter.  As they say, "Next!".
The default limit has gone up over the years.  It used to be around 2000, now is well over 5000.

As to my test program the 103ish threads are sleeping most of the time.

If you want my code to try, just shout - it will probably run even if you can't implement a couple of the system calls - you can monitor the memory and thread count via the OS status panel (whatever it's called).  250 lines - best compiled and run from monitor.