Recent

Author Topic: MultiThread Programming, more Thread not always faster.  (Read 945 times)

incendio

  • Jr. Member
  • **
  • Posts: 92
MultiThread Programming, more Thread not always faster.
« on: June 19, 2019, 05:45:05 am »
Hi guys,

I have 8 threads that assign to 5 jobs, job 1 &  job 2 each one, assign 1 thread, while job 4-5, each one assign 2 threads,see picture 1 (Th1.png).

Job 1 is the lightest, so its thread finished first. Job 5 is the heaviest one.

I want to finished the jobs faster, so I divided job 5 into 3 sections and plan to assign 3 threads to this job, but not assign all three threads immediately, first assign 2 threads to process section 1 & 2, and then, when thread from job 1 finished, assign another thread to process section 3 in job 5, see picture 2 (Th2.png).

I thought by adding more threads to job5, all jobs will finished faster, but it was not, in fact, it was slower by a couple of seconds.

Here are the thread declaration
Code: Pascal  [Select]
  1. TGetDtThr = class(TThread)
  2.   private
  3.   public
  4.     IdLoc: string;
  5.     Range1,Range2:integer;
  6.     Name:string;
  7.     constructor Create(CreateSuspended: boolean; AThreadDone: TNotifyEvent); overload;
  8.   protected
  9.     procedure Execute; override;
  10. end;                          

Threads codes
Code: Pascal  [Select]
  1. constructor TGetDtThr.Create(CreateSuspended: boolean; AThreadDone: TNotifyEvent);
  2. begin
  3.   inherited Create(true);
  4.   FreeOnTerminate := true;
  5.   OnTerminate := AThreadDone;
  6.   ThreadsRunning := ThreadsRunning + 1;
  7.   MainFrm.LblThr.Caption := IntToStr(ThreadsRunning);
  8. end;
  9.  
  10. procedure TGetDtThr.Execute;
  11. begin
  12.   mainfrm.StartThread(IdLoc,Range1,Range2);
  13. end;
  14.  

Codes to execute picture 1 in MainFrm.pas
Code: Pascal  [Select]
  1. procedure TMainFrm.Go();
  2. var
  3.   Dt1,Dt2,Dt3,Dt4,Dt5,Dt6,Dt7,Dt8 : TGetDtThr;
  4. begin
  5.   Dt1         := TGetDtThr.Create( True,@ThreadDone);
  6.   Dt1.IdLoc   := 'Job1';
  7.   Dt1.Range1  := 1;
  8.   Dt1.Range2  := 30;
  9.   Dt1.Name    := 'T1';
  10.   Dt1.Start;
  11.  
  12.   Dt2         := TGetDtThr.Create( True,@ThreadDone);
  13.   Dt2.IdLoc   := 'Job2';
  14.   Dt2.Range1  := 1;
  15.   Dt2.Range2  := 30;
  16.   Dt2.Name    := 'T2';
  17.   Dt2.Start;
  18.  
  19.   Dt3         := TGetDtThr.Create( True,@ThreadDone);
  20.   Dt3.IdLoc   := 'Job3';
  21.   Dt3.Range1  := 1;
  22.   Dt3.Range2  := 15;
  23.   Dt3.Name    := 'T3';
  24.   Dt3.Start;
  25.  
  26.   Dt4         := TGetDtThr.Create( True,@ThreadDone);
  27.   Dt4.IdLoc   := 'Job3';
  28.   Dt4.Range1  := 16;
  29.   Dt4.Range2  := 30;
  30.   Dt4.Name    := 'T4';
  31.   Dt4.Start;
  32.  
  33.   Dt5         := TGetDtThr.Create( True,@ThreadDone);
  34.   Dt5.IdLoc   := 'Job4';
  35.   Dt5.Range1  := 1;
  36.   Dt5.Range2  := 15;
  37.   Dt5.Name    := 'T5';
  38.   Dt5.Start;
  39.  
  40.   Dt6         := TGetDtThr.Create( True,@ThreadDone);
  41.   Dt6.IdLoc   := 'Job4';
  42.   Dt6.Range1  := 16;
  43.   Dt6.Range2  := 30;
  44.   Dt6.Name    := 'T6';
  45.   Dt6.Start;
  46.  
  47.   Dt7         := TGetDtThr.Create( True,@ThreadDone);
  48.   Dt7.IdLoc   := 'Job5';
  49.   Dt7.Range1  := 1;
  50.   Dt7.Range2  := 15;
  51.   Dt7.Name    := 'T7';
  52.   Dt7.Start;
  53.  
  54.   Dt8         := TGetDtThr.Create( True,@ThreadDone);
  55.   Dt8.IdLoc   := 'Job5';
  56.   Dt8.Range1  := 16;
  57.   Dt8.Range2  := 30;
  58.   Dt8.Name    := 'T8';
  59.   Dt8.Start;
  60. end;
  61.  
  62. procedure TMainFrm.ThreadDone(Sender: TObject);
  63. begin
  64.   ThreadsRunning := ThreadsRunning-1;
  65.   lblThr.Caption:= IntToStr(ThreadsRunning);
  66.  
  67.   if (ThreadsRunning <= 0) then
  68.   begin
  69.      ShowMessage('Finished');
  70.   end;
  71. end;
  72.  

Codes to execute picture 2.
The different is on procedure Go in Mainfrm, it look like these :
Code: Pascal  [Select]
  1.   Dt7         := TGetDtThr.Create( True,@ThreadDone);
  2.   Dt7.IdLoc   := 'Job5';
  3.   Dt7.Range1  := 1;
  4.   Dt7.Range2  := 10;  // -> noticed the range
  5.   Dt7.Name    := 'T7';
  6.   Dt7.Start;
  7.  
  8.   Dt8         := TGetDtThr.Create( True,@ThreadDone);
  9.   Dt8.IdLoc   := 'Job5';
  10.   Dt8.Range1  := 11; // -> noticed the range
  11.   Dt8.Range2  := 20; // -> noticed the range
  12.   Dt8.Name    := 'T8';
  13.   Dt8.Start;

Also, procedure threard done, now look like these :
Code: Pascal  [Select]
  1. procedure TMainFrm.ThreadDone(Sender: TObject);
  2. begin
  3.   ThreadsRunning := ThreadsRunning-1;
  4.   lblThr.Caption:= IntToStr(ThreadsRunning);
  5.  
  6.   if (ThreadsRunning <= 0) then
  7.   begin
  8.      ShowMessage('Finished');
  9.   end
  10.   else
  11.   begin
  12.      if (TGetDtThr(Sender).Name = 'Ret') then
  13.      begin
  14.         Go2();
  15.      end;
  16.   end;
  17. end;
  18.  

Procedure Go2
Code: Pascal  [Select]
  1. procedure TMainFrm.Go2();
  2. var
  3.   Dt3 : TGetDtThr;
  4. begin
  5.   // Loc3
  6.   Dt3         := TGetDtThr.Create( True,@ThreadDone);
  7.   Dt3.IdLoc   := 'Job5';
  8.   Dt3.Range1  := 21;
  9.   Dt3.Range2  := 30;
  10.   Dt3.Name    := 'T1';
  11.   Dt3.Start;
  12. end;
  13.  

Is there a guide about how many threads should use to achieve maximum performance?

Peter H

  • Jr. Member
  • **
  • Posts: 57
Re: MultiThread Programming, more Thread not always faster.
« Reply #1 on: June 19, 2019, 06:59:53 am »
If the threads dont have pending IO waiting phases, then there is no to little speedup.
If e.g. the CPU has 4 cores then a speedup more than factor 4 is not possible.

Each thread might have its own processor and so they /can/ run in parallel.
On threadswitch the content of cache most probably becomes invalid and is lost.
If the threads work on large datasets that do not fit into the cache, they cannot truly run in paralel, because they must share the memory bandwith. In this case the program slows down.
« Last Edit: June 19, 2019, 07:10:51 am by Peter H »

PascalDragon

  • Hero Member
  • *****
  • Posts: 726
  • Compiler Developer
Re: MultiThread Programming, more Thread not always faster.
« Reply #2 on: June 19, 2019, 08:55:01 am »
@incendio: You forgot the most important part: what are the threads doing inside Mainfrm.StartThread? Depending on the code and the locking involved it could be that the threads block each other quite much and then there won't be a big improvement in performance then.

incendio

  • Jr. Member
  • **
  • Posts: 92
Re: MultiThread Programming, more Thread not always faster.
« Reply #3 on: June 19, 2019, 08:58:18 am »
@incendio: You forgot the most important part: what are the threads doing inside Mainfrm.StartThread? Depending on the code and the locking involved it could be that the threads block each other quite much and then there won't be a big improvement in performance then.
It was connecting to Firebird Database, run stored procedure and then do some insert to database. Actually most of the process done by stored procedure in Firebird, no  processing in the thread.

On my CPU with 8 cores, maximum performance was reach when no of threads equal to 75% of cores, means 6 threads, more than that has no effect or even got poor performance.

This is very general, need more test, perhaps, others users can share their experiences.
« Last Edit: June 19, 2019, 09:00:54 am by incendio »

Thaddy

  • Hero Member
  • *****
  • Posts: 9293
Re: MultiThread Programming, more Thread not always faster.
« Reply #4 on: June 19, 2019, 09:05:58 am »
Under windows you can associate the threads with a lighter work load to the same single core. That will speed things up:
See https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-setthreadaffinitymask and https://en.wikipedia.org/wiki/Affinity_mask

Heavy load threads should run on a different dedicated single core too.

It is also possible on unix, but I never had the need, so I haven't got the information present. (16-64 core blades...)
« Last Edit: June 19, 2019, 09:18:06 am by Thaddy »
also related to equus asinus.

incendio

  • Jr. Member
  • **
  • Posts: 92
Re: MultiThread Programming, more Thread not always faster.
« Reply #5 on: June 19, 2019, 09:16:51 am »
I am run Lazarus on Linux mint, so no Windows here.

Just for info, Firebird 3 on Linux Mint run faster about 15%-20% compared to Windows 10.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 635
Re: MultiThread Programming, more Thread not always faster.
« Reply #6 on: June 19, 2019, 09:28:59 am »
As said, the things you want to avoid are serializing (locking and shared memory) and waiting for completion. If the calculation is not a one-shot affair, the best way to do it is generally by using free-running tasks: have the tread start a new task when it is ready. But you're doing it pretty good already.

Next up is: profiling. Start with the database, then log start and end times. Or use a profiler. There is probably only one thing that takes most of the time.

incendio

  • Jr. Member
  • **
  • Posts: 92
Re: MultiThread Programming, more Thread not always faster.
« Reply #7 on: June 19, 2019, 11:42:32 am »
The best performance I can get is running 6 Threads, see picture.

All jobs finished in about 42 secs.

The heaviest job, job5, if run directly on firebird sql, took about 38 secs.


rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: MultiThread Programming, more Thread not always faster.
« Reply #8 on: June 19, 2019, 12:08:56 pm »
The heaviest job, job5, if run directly on firebird sql, took about 38 secs.
Maybe also look into the stored procedure.
With some fiddling maybe you can cut that 38 down to just a few seconds using the correct indexes etc.

(You might be doing a lot work trying to get this into threads while the majority of the work is done on the server which could be optimized too)

incendio

  • Jr. Member
  • **
  • Posts: 92
Re: MultiThread Programming, more Thread not always faster.
« Reply #9 on: June 19, 2019, 01:32:03 pm »
The heaviest job, job5, if run directly on firebird sql, took about 38 secs.
Maybe also look into the stored procedure.
With some fiddling maybe you can cut that 38 down to just a few seconds using the correct indexes etc.

(You might be doing a lot work trying to get this into threads while the majority of the work is done on the server which could be optimized too)

Thanks, but that's not my point. My point is to find out how's multithreading performance on Lazarus.

This multithreading performance is great. Since firebird doesn't support parallelization query, those jobs if only doing by firebird server must execute one by one, it will tooks at least 120 seconds to complete all job, with threads, cut of about 70%.