Hello,
Now i am working on my Parallel compression library and finishing it, it will support parallel LZMA, parallel BZIP ,
parallel Gzip and parallel LZ... and since my parallel compression library is using the IO a lot, i am overlapping
the IO processing and CPU processing , this is making my library more efficient and faster... and I will finish it soon..
Like in superscalar pipeline stages , i am overlapping the IO processing and CPU processing and my Parallel compression library is giving me a very good performance in my benchmarks
I am using a lockfree_queue between the the worker threads producers that compresses the data and the worker thread that does IO processing and copy the data to disk..
And of course since my Parallel library is 'more' IO bounded you have to have a faster harddisk (Raid etc..)
When i have compressed from a memory stream to memory stream it does give a lighting speed (its very fast on an in-memory compression..)
You will see my full version soon that supports parallel LZMA, parallel BZIP , parallel Gzip and parallel LZ... and of course, it does uses my Threadpool engine that uses work-stealing and a lock-free worker queue for each worker thread in the threadpool..
Regards,
Amine Moulay Ramdane.
Recent
