Recent

Author Topic: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64  (Read 11927 times)

Max V. Terentiev

  • New Member
  • *
  • Posts: 30
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #15 on: August 07, 2017, 12:32:31 pm »
Quote
On CentOS6, the file /etc/security/limits.d/90-nproc.conf has a soft limit (soft nproc 1024) that override that setting.

I was 4096 here. Changed to 65535. But it's not helps (

Blestan

  • Sr. Member
  • ****
  • Posts: 461
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #16 on: August 07, 2017, 12:58:05 pm »
look anyway the problem is not fpc/lazarus or synapse/indy releated the problem is the multi-threading support on linux ...
try to make a non  socket program in c/c++ with 1000+ threads and you will get the same craches ...
1000+ threads on 8 core cpu is insane and counter productive  >:D >:D >:D
Speak postscript or die!
Translate to pdf and live!

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #17 on: August 07, 2017, 01:19:46 pm »
Unless all those threads are listening on a socket in your full application, I agree: just do it serial. Much faster as well.

Max V. Terentiev

  • New Member
  • *
  • Posts: 30
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #18 on: August 07, 2017, 01:48:28 pm »
Quote
1000+ threads on 8 core cpu is insane and counter productive

Of course I not use 1000+ threads in my app ! I use only 100-150.

But. This problem may happens on 100-150 threads after several hours.

So, to find and fix bug I write test app with 1000+ threads where problem happens in several seconds. It's just to avoid waiting for hours during debugging )

Anyway, with 1000 or 5000 threads apps must works stable if OS have enouch resources. App may work very slow or ineffective, use to much memory, etc but it's MUST works stable, right ? )

Max V. Terentiev

  • New Member
  • *
  • Posts: 30
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #19 on: August 07, 2017, 01:51:40 pm »
Quote
look anyway the problem is not fpc/lazarus or synapse/indy releated the problem is the multi-threading support on linux ...

You mean multi-threading support in Linux OS ? Or multi-threading support in FPC->Linux targets ?

OS should not have any problems with threads. For example, nginx can serve thousands requests with no problems.

Blestan

  • Sr. Member
  • ****
  • Posts: 461
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #20 on: August 07, 2017, 01:59:14 pm »
i repeat for the 10th time this is NOT fpc problem!!!!!
nginx is single threaded async io based and does not use more than the main thread  >:D >:D >:D Please read.alttle bit more about socket programing!!!!!!
Speak postscript or die!
Translate to pdf and live!

Max V. Terentiev

  • New Member
  • *
  • Posts: 30
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #21 on: August 07, 2017, 02:04:55 pm »
Ok, how many threads is safe and 100% stable on Linux ?

Blestan

  • Sr. Member
  • ****
  • Posts: 461
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #22 on: August 07, 2017, 02:18:59 pm »
im using 8 or 16  depending the core... also tested up to 64 just for the benchmarks ... but my code is very tight not even doing memory allocation and string creation... every thing is using mem pools per thread.i get under 2ms per simple http request/reponse so for 8 cores it runs at 5-20k requests
« Last Edit: August 07, 2017, 02:23:06 pm by Blestan »
Speak postscript or die!
Translate to pdf and live!

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #23 on: August 07, 2017, 02:21:10 pm »
The optimal amount of active threads (ie. threads that are running code) is twice the amount of cores. Three threads running at full speed on each core will make the computer unresponsive until it crashes.

That is not OS specific, it is purely hardware related.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1313
Re: 1000+ threads and Synapse or Indy = VERY unstable on Linux 64
« Reply #24 on: August 09, 2017, 09:56:44 pm »
Ok, to explain this a bit more:

1. It is very hard to get threads to run at full speed, because they're probably waiting a long time to read or write stuff to and from the slow memory.
2. Intel tells us, that hyper-threading effectively doubles the amount of threads you can run.
3. The Windows threading model and the *nix one are vastly different.

A long time ago (in a galaxy far away?) we had Unix, which was multi-user and multi-tasking, and Windows 95, which only pretended to be those things. And, starting a process in Windows takes a long time. So, Microsoft went the way of the least resistance, and made it possible to just start threads at random, without any support structure whatsoever. "Now we are multi-whatever as well!!!"

The *nix guys had to laugh at that: they passed that stage more than twenty years ago. It wouldn't work. You need separation and messages to make it work. And they were right.

Then again, when Microsoft spends many millions on promoting it, who are you going to believe? Those old nerds? Nah, of course not! Microsoft is the winning team! Jump on the bandwagon!

So, for twenty years, people believed Microsoft, and tried to get it to work. And if they succeeded, profiling was not the way to go in most cases, as it showed that their multi-threaded application was actually slower than the single-threaded one. WTF!!!

And, of course, Linux now has threads and all the other crap as well: we have to copy the winning team!


Nowadays, most programmers who use multi-whatever a lot, have taken a step back and they don't use most of the things Microsoft tells them to. Like, shared memory. Or keeping pointers to other threads. Or, actually the whole programming model as they learned it on school. Because, it doesn't work!

Then again, how do you do it, in a way that it actually works? Can I get an education for that? Not yet, probably. Or, not any more.


So, when I made a fast, working, multi-platform threading library a few years ago (in C++, unfortunately), I did have to figure it all out myself. And test it as well. And I can state, that my testing showed, that both on ARM and Intel processors, in multiple variants and on multiple platforms (Windows, Linux, OS-X, iOS and Android), two threads for each core, that are designed to run at full speed (they use very little memory) are fine, while three will crash the whole box after a while.

But, as stated, it is very hard to make useful threads/processes that do more useful things than just benchmarking, that run at full speed. Most of those threads/processes spend the majority of their time idle/suspended/waiting for memory or I/O.
« Last Edit: August 10, 2017, 10:36:46 pm by SymbolicFrank »

 

TinyPortal © 2005-2018