Forum > Beginners

5000 threads

(1/3) > >>

please advise if it would be an issue with Lazarus creating 5000 threads simultaneously (on ubuntu) for devops purposes aka checking services on the infrastructure.
i understand it is about 10 Gb RAM but RAM is cheap nowadays. (i do not know if i want to work with a thread pool at 5000 and maintain that amount of RAM occupied)
if the number is an issue, what would be the maximum that is stable.
thank you

Threads are more CPU issue. Howw many Cores have the machine ? The CPU and the Sytemmust switch between the processes = Threads

I think 5000 Threads sounds like a bad design.


Well there is a little more to it. In principle 5000 threads is not such a big problem (some webservers are in that ball park).
There are some issues that you need to take care of:
- Thread stack allocation: On Windows, the default is rather huge. Use the compiler switches to limit it.
- Context switching: try to avoid that with such a number of threads. Do NOT let the threads interact with each other: information sharing is a bad idea with so many threads.
- CPU cores and virtual cores: How many do you have? Devide 5000/cpu cores and using fibers per core or virtual core is usually more efficient.

Why not? As they are only checking services they are idle most of the time, arent't they?

You can also limit them to only run a specfied number simultaneously by using a semaphore.
So if you have a CPU with 8 threads you can limit them to 8 simultaneoiusly running.

from a requirements perspective i have to check that let us say (x > 100):
- x database instances have upgraded correctly by checking their version;
- x database instances are up every hour by running some SQL script;

so from a solution stand point what are my options?


[0] Message Index

[#] Next page

Go to full version