Recent

Author Topic: My threadpool engine and scalability [1]...  (Read 2205 times)

aminer

  • Hero Member
  • *****
  • Posts: 956
My threadpool engine and scalability [1]...
« on: October 01, 2012, 07:01:24 pm »

Hello,

I have corrected my mistakes , please read again:

If you take a look at my threadpool engine:

http://pages.videotron.com/aminer/

you will notice that it does not scale in the execute() method
cause LockedIncLong(balance1) is expensive.

here is the execute method:

==

function TThreadPool.execute(func:TMyProc;const Context: Pointer): Boolean;

var
local_balance,local_count:long;

params: PParams;
p:pointer;

begin
new(params);
setlength(params^,1);
params^[0].proc:=func;
params^[0].data:=context;

local_balance:=LockedIncLong(balance1) mod FThreadCount;
while not Queues[local_balance].push(tobject(params)) do sleep(0);
events[local_balance].setevent;
end;

===

Now to be able to scale the execute() method we have to add a new
execute1() method that will be called from mutiple threads, i have to
use LockedIncLong() but it will be called only once:

==

function TThreadPool.execute1(func:TMyProc;const Context:
Pointer;index:long): Boolean;

params: PParams;
p:pointer;

begin
new(params);
setlength(params^,1);
params^[0].proc:=func;
params^[0].data:=context;

if index = 0
then
begin
index:=LockedIncLong(balance1) mod FThreadCount;
end;

while not Queues[index].push(tobject(params)) do sleep(0);
events[index].setevent;
end;

=


so we have to call the execute1() method from mutiple threads like this:

and we have to initialize index to zero before:, like this:

ThreadPool.execute1(myproc,data,index)

I am using mutiple lockfree FIFO queues inside my threadpool engine,
so that there is less contention and using work-stealing also, so after
adding execute1() methods , this threadpool engine will become scalable.
this threadpool engine can also be used as a scalable queue
(not a strict FIFO queue, but it's also useful).

This threadpool engine is useful and that is what's important.



I will try to update my threadpol engine soon.



Thank you,
Amine Moulay Ramdane.



aminer

  • Hero Member
  • *****
  • Posts: 956
Re: My threadpool engine and scalability [1]...
« Reply #1 on: October 01, 2012, 07:36:20 pm »

Hello,

As you will notice, inside the new exucuted1() i have used the same method
as with my implementation of the Distributed Reader-Writer Mutex 1.04
and i am using a variable number of mrews to lower the contention
more in my implementation of Distributed Reader-Writer Mutex 1.04
based on the Dmitry Vyukov C++ Distributed Reader-Writer Mutex,
and as i said i am using mutiple lockfree FIFO queues inside my threadpool engine,
so that there is less contention and using work-stealing also, so after adding the execute1()
method , this threadpool engine will become scalable and this threadpool engine can also
be used as a scalable queue(not a strict FIFO queue, but almost, and it will be also useful).

This threadpool engine is useful and that is what's important.

I will try to update my threadpol engine soon.

You can download Distributed Reader-Writer Mutex and my other paralle libraries
from:

http://pages.videotron.com/aminer/




Amine Moulay Ramdane.



aminer

  • Hero Member
  • *****
  • Posts: 956
Re: My threadpool engine and scalability [1]...
« Reply #2 on: October 01, 2012, 08:06:30 pm »

I have wrote my parallel libraries with love and i have
given them with love , my libraries look like the following
song of Bob Marley, and i am a socialist in the deep of
my soul and my heart:

It's a socialist soldier in the heart of America !


http://www.youtube.com/watch?v=S5FCdx7Dn0o




Amine Moulay Ramdane.


 

TinyPortal © 2005-2018