Recent

Author Topic: Parallelcompression library was updated to version 2.16 ...  (Read 3480 times)

aminer

  • Hero Member
  • *****
  • Posts: 956


Hello,

Parallelcompression library was updated to version 2.16 , i have
just changed the code inside my threadpool.pas engine inside the
parallel compression zipfile to not use work-stealing , cause work-stealing
will cause a deadlock in parallelcompression library, so please use the
threadpool engine that i have provided you inside the parallelcompression zipfile.

Description:

Parallel compression library implements Parallel Gzip , Parallel Bzip ,
Parallel LZMA and Parallel LZ algorithms using my Thread Pool Engine.


You can download my ParallelCompression  library version 2.16 from:

http://pages.videotron.com/aminer/




Thank you.

Amine Moulay Ramdane.



aminer

  • Hero Member
  • *****
  • Posts: 956
Re: Parallelcompression library was updated to version 2.16 ...
« Reply #1 on: May 20, 2012, 12:27:40 am »

Hello,

I will try to clarify more this problem that i have solved in
version 2.16 ...

If you take a look inside the parallelcompression zipfile,
inside the file parallelgzip.pas, you will notice that i am using a threadpool
engine and inside the TCallbacks.GzipCompress() method  i am using 
GZCompressStream() outside the critical section , this is good for
scalability, but inside the same method i am using also a critical section
like this:

repeat
If TParallelGzip(Tjob(obj).obj).count_compress = Tjob(obj).index
then
 begin
  // i am using some code here
  Inc(TParallelGzip(Tjob(obj).obj).count_compress);
  break;
 end;
sleep(0);
until false;

as you have noticed , in this critical section  i am testing like this:
 If TParallelGzip(Tjob(obj).obj).count_compress = Tjob(obj).index
and if you read carefully the code inside parallelgzip.pas you will
notice that since i am using a critical section like this , and using
a threadpool engine, the threadpool engine must not use work-sealing
or this will cause a deadlock inside the TCallbacks.GzipCompress() method,
so to correct this problem i had to use for example a FIFO queue for
each worker thread to enhance the scalability and completly avoid
work-stealing (from the LIFO Stacks.or FIFO Queues).


Thank you for your time.



Amine Moulay Ramdane. 



 

TinyPortal © 2005-2018