Amine's announcements

[aminer]

Hello


Parallel Compression 1.0


Author: Amine Moulay Ramdane


Description:

Parallel Zlib and Parallel Bzip algorithms that use my Thread Pool Engine.


Please look at pzlib.pas and pbzip.pas inside the zip file - compile and execute them...


And please see the benchmarks here:

http://pages.videotron.com/aminer/ParallelCompression/parallelbzip.htm


Language: FPC Pascal v2.2.0+ / Delphi 7+: http://www.freepascal.org/


Operating Systems: Win , Linux and Mac (x86).


Required FPC switches: -O3 -Sd -dFPC -dWin32 -dFreePascal

-Sd for delphi mode....

Required Delphi switches: -DMSWINDOWS -$H+

For Delphi 5,6,7 use -DDelphi

For Delphi 2005,2006,2007,2009,2010+ use the switch -DDELPHI2005+




Sincerely,
Amine Moulay Ramdane.

[aminer]

Hello,

Plese read more here:

http://software.intel.com/en-us/forums/showthread.php?t=73269&o=d&s=lr


Sincerely,
Amine Moulay Ramdane.

[aminer]

Hello all,


My Lock-free ParallelQueue algorithm have been updated
- my new algorithm avoids false sharing - , the new version
have scored 17 millions of pop()  transactions per second
- the old score was 7 millions - on an  Intel Core 2 Quad Q6600


There are three ways to reduce lock contention:

1- Reduce the duration for which locks are held
2- Reduce the frequency with which locks are requested

or

3- Replace exclusive locks with coordination mechanisms that
   permit greater concurrency.

With low , moderate AND high contention, my ParallelQueue algorithm
offers better scalability - and i am using it inside my Thread Pool Engine -


Because my lock-free ParallelQueue algorithm uses a hash based
method - and lock striping -  and using just a LockedInc() , so, i am
REDUCING the duration for which locks are held AND REDUCING the
frequency with which locks are requested,  hence i am REDUCING A LOT
the contention, so it's very efficient.

And i can state the following law or theorem:


[1]  If there is LESS contention THEN  the algorithm will scale better.
     Due to the fact that S (the serial part) become smaller with less contention ,
     and as N become bigger,   the result - the speed of the program/algorithm...
     -  of the Amdahl's equation  1/(S+(P/N)) become bigger.

So, since my lock-free ParallelQueue algorithm is REDUCING A LOT
the contention,  it's very efficient and it scales well..

Also, i can state another law or theorem like this:


[2]  If there is false sharing THEN  the algorithm will not scale well. Due to
      the fact that S (the serial part) of the Amdahl's equation  1/(S+(P/N)) will
      become bigger, so, this is not good for scalability.

It's why i am also avoiding false sharing in my lock-free ParallelQueue algorithm.

So, my new lock-free ParallelQueue algorithm reduces a lot the contention
and  avoids false sharing, it's why it scored 17 millions of pop() transactions
per second... better than flqueue and RingBuffer.

Please look at the benchmarks here:

http://pages.videotron.com/aminer/parallelqueue/parallelqueue.htm

http://pages.videotron.com/aminer/



Sincerely,
Amine Moulay Ramdane.

[aminer]

Hello

Parallel Quicksort using two cores, it is for educational purpose...

It uses my Thread Pool Engine and quicksort two arrays in parallel
and merge them....

Note: On Delphi it gave me 1.7x on two cores - with more optimization,
delphi will give you better scalability...- . Freepascal gave me just 1.2x,
so, FreePascal still have to be optimized on 'some' parts for better parallel
computing...

And please look at pqsort.pas inside the zip file - compile and execute it...


You can download Parallel Quicksort from:

http://pages.videotron.com/aminer/


Language: FPC Pascal v2.2.0+ / Delphi 7+: http://www.freepascal.org/


Operating Systems: Win , Linux and Mac (x86).


Required FPC switches: -O3 -Sd -dFPC -dWin32 -dFreePascal


-Sd for delphi mode....


Required Delphi switches: -DMSWINDOWS -$H+


For Delphi 5,6,7 use -DDelphi


For Delphi 2005,2006,2007,2009,2010+ use the switch -DDELPHI2005+


Sincerely,
Amine Moulay Ramdane.

[aminer]

Hello


Parallel Compression library version 1.2


Author: Amine Moulay Ramdane


Description:


Parallel Gzip , Parallel Zlib and Parallel Bzip algorithms that uses my
Thread Pool Engine.


And please look at the test examples inside the zipfile:
test_pgzip.pas , pzlib.pas , test_pbzip.pas - compile and execute them... -


And please see the benchmarks here:


http://pages.videotron.com/aminer/ParallelCompression/parallelbzip.htm

You can download my Parallel Compression library from:

http://pages.videotron.com/aminer/


The Parallel Compression library is VERY easy to use , and
here is an example in Object Pascal: - it's a parallel gzip compression -


--------------------------

program test;


uses ParallelCompression,classes,zlibex,sysutils;

var
pgzip:TParallelGzip;
fstream1,fstream2,fstream3:TFileStream;
name:string;

begin

name:='msvcr100.dll';  // set your your test file ...

pgzip:=TParallelGzip.create(4); // number of cores...

fstream1:=TFileStream.create(name, fmOpenRead);//+fmShareExclusive);
fstream2:=TFileStream.create(name+'.gz',fmCreate);//+fmShareExclusive); // destination of the compression...
fstream3:=TFileStream.create(name+'.ok', fmCreate);//+fmShareExclusive); // destination decompression...


pgzip.compress(fstream1,fstream2,zclevel9); // select a compressionlevel from
                                           // zcNone or zcLevel1 to zcLevel9
   

writeln;
writeln('Parallel compression finished...');

fstream2.position:=0;

pgzip.decompress(fstream2,fstream3);

writeln;
writeln('decompression finished...');


fstream1.free;
fstream2.free;
fstream3.free;
pgzip.free;

end.

-----------------------------------------------


Language: FPC Pascal v2.2.0+ / Delphi 7+: http://www.freepascal.org/


Operating Systems: Win , Linux and Mac (x86).


Required FPC switches: -O3 -Sd -dFPC -dWin32 -dFreePascal


-Sd for delphi mode....


Required Delphi switches: -DMSWINDOWS -$H+


For Delphi 5,6,7 use -DDelphi


For Delphi 2005,2006,2007,2009,2010+ use the switch -DDELPHI2005+

 


Sincerely,
Amine Moulay Ramdane.

 

[aminer]

Hello,


As you have noticed , my Parallel Compression library 
works on TStreams class..

That means it supports TFileStream(files..) or TMemoryStream(memory...) ...
of Object Pascal.


Welcome: http://pages.videotron.com/aminer/

Have fun !   


Take care...

Sincerely,
Amine Moulay Ramdane.

[faber]

What is the license for your code? is there any?
I see that you attach a lot of files with various license headers, like:
-Borland Delphi Supplemental Components
-Y Library Public License Version
-Borland Delphi Visual Component Library
-BSD etc.

just for information and education?

[aminer]

Hello,

>I see that you attach a lot of files with various license headers, >like:-Borland Delphi Supplemental Components

It's only classes.pas that i have patched for delphi5
support and renamed it Classespatch.pas.

If you don't want delphi 5 support just delete it...

>-Y Library Public License Version

No problem with this...

I have also modified zlibex.pas to be able
to dynamicaly link to  zlib1.dll  or zlib1.so
for portability ...



Amine.

[aminer]

I wrote:
>I have also modified zlibex.pas to be able
>to dynamicaly link to  zlib1.dll  or zlib1.so
>for portability ...


I have included the last version of zlib1.dll in
the zipfile...

You can also download zlib1.dll and zlib1.so from:

http://www.zlib.net/


Amine.

[aminer]

Hello,

As i said , i have modified zlibex.pas etc.

But you can find those stupid zlibex.pas...on internet:

http://devemanage.googlecode.com/svn-history/r182/trunk/Source/Class


Sincerely,
Amine.

[aminer]

Hello,

I have tried PasZlib: http://sageshome.net/oss/paszlib-sg.php

but Paszlib is not so fast and efficient...

So, i have modified Zlibex.pas ... and prefered to use
the dynamic library zlib1.dll and zlib1.so ... that you
find on: http://www.zlib.net/


Sincerely,
Amine.

[aminer]

Hello,


I have just updated parallel quicksort , now you can parallel quicksort
on 'many' cores - not just two cores -  look at the new program
pqsort2.pas inside the zip 

Next step i will convert pqsort2.pas program to a unit that you can import...

I am using inside pqsort2.pas the following functions:
QSortStr() , QuickSrtInt() and Merge()

Parallel quicksort version 1.01 quicksort many array parts - of your
array -  in parallel using Quicksort, and after that it finally merge
them - with the merge() procedure -

And as you know , Quicksort is a divide and conquer algorithm that
have the following best case performance:
 
T(n) = T(n/2) + T(n/2) + O(n)
       = 2T(n/2) + (n)

And from case 2 of Master theorem, the recurrence equation gives:

        T(n) = (n lg n)


Now, i have tested mergesort in parallel , but quicksort() gives better
performance. 


Note also that on Delphi parallel quicksort gave me 1.7x on two cores
- but with more optimization, delphi will give you better scalability...- .
 
Freepascal on the other hand gave me just 1.2x on two cores, so,
FreePascal still have to be optimized on 'some' parts for better parallel
computing...


You can download parallel quicksort from:

http://pages.videotron.com/aminer/


Sincerely
Amine Moulay Ramdane.

[aminer]

Hello,


Description:


Why the  2 Go limitation on the Win 32bits systems ?


My AWE object pascal module is here, it  allows your application
 to use up to 64GB of RAM.


And here is the public interface of TAWEStorage class,
i  have implemented the follow methods:


TAWEStorage = class(TObject)
  public


    Size:longword;
    position:longword;
    PageSize:integer;


    constructor Create();
    destructor Destroy(); override;
    function GetMem(ulBytes: ULONG): BOOL;
    function FreeMem(): BOOL;
    function HowManyPagesAllocated(): ULONG;
    function CopyFrom(in_stream:TMemoryStream;count:longint):longint;
    function CopyTo(out_stream:TMemoryStream;count:longint):longint;
    function Read(var Buffer;count:longint):longint;
    function Write(const Buffer;count:longint):longint;
    function ToString:string;


  end;


Note: To be able to use AWE you have to set the user rights
correctly,
so,go to:  Control Panel -> Administrative tools -> Local Security
Policy
-> User Rights Assignment and give 'Lock pages in memory' right
to the user that wants to use AWE.


Every TAWStorage object can go up to 4GB , and you can go up to
64GB !


You can download my AWE module from:

http://pages.videotron.com/aminer/


TAWEStorage is very easy to use, here is an example:


-------------------------------------------------


program test;


uses AWE,classes;


var
awe1:TAWEStorage;
ptr:pointer;
fstream1:TFileStream;
stream1,stream2:TMemoryStream;


Function StringToStream(const AString: string): TMemoryStream;
begin
Result := TMemoryStream(TStringStream.Create(AString));
end;


begin


stream2:=TMemoryStream.create;


fstream1:=TFileStream.create('test.txt',fmCreate);


 stream1:=StringToStream('Hello world !');
awe1:=TAWEStorage.create;


writeln;
if awe1.getmem(stream1.size)
  then writeln('Memory was reserved...')
  else writeln('Memory was not reserved...');


writeln('Number of pages allocatd is: ',awe1.HowManyPagesAllocated);


if awe1.CopyFrom(stream1,stream1.size) <> 0
then writeln('CopyFrom ok ...')
else writelN('CopyFrom not ok ..');


awe1.position:=0;
awe1.CopyTo(stream2,awe1.size);


stream2.position:=0;
fstream1.copyfrom(stream2,stream2.size);
stream2.position:=0;
fstream1.position:=0;


writeln(awe1.tostring);


awe1.freemem;
awe1.free;
end.


--------------------------------------------------------------




Sincerely,
Amine Moulay Ramdane.

[Vincent Snijders]

I merged this announcement with your other announcements.

[aminer]

Hello,


There is another module at:

http://www.winsoft.sk/awe.htm


But you have to pay for it.. and i think my module is *better*.


Welcome: http://pages.videotron.com/aminer/


Note: My module is a nice module that you can use for
        example with ParalleHashList...


Sincerely,
Amine Moulay Ramdane.