ParalellVarFiler version 1.17

[aminer]

Hello again my dear FreePascal and Lazarus community..


Question:

Amine, and what about your scalable RWLock that you have implemented ?

here it is:

http://pages.videotron.com/aminer/rwlock1.html


You have to understand something, if
you take a look at OmniThread from Gabr that was implemented for the Delphi community , here it is:

http://otl.17slon.com/


If you download the zipfile of OmniThread
and you take a look at the lockfree RWLock that Gabr wrote and that is called TMREW, you will notice that this RWLock is using a CAS on a global variable, hence this is expensive , so if
you want TMREW to scale, the time of the RWLock's READ section must be bigger that the time to
execute a those CASes on a global variable with cache misses, and that's not good, so this
is why i have implemented my scalable RWLock that scales even if the RWLOCK's READ section is small.

You can dowload my scalable RWLock from:

http://pages.videotron.com/aminer/rwlock1.html


Thank you,
Amine Moulay Ramdane,.

[aminer]

Hello,

I have to be more precise...

I said that the TMREW from Omnithread
is not good cause the time of the TMREW  reader section must be bigger that the time to execute a those CASes on a global variable with cache misses,
more than that since the TMREW is using
a CAS on a global variable on the reader side also, hence it will generate too much coherence-traffic under contention , so this why TMREW from Omnithread is not scalable. But my scalable RWLock is scalable and it minimizes efficiently the cache-coherence traffic.


You can download my scalable RWLock from:

http://pages.videotron.com/aminer/



Thank you,
Amine Moulay Ramdane.

[aminer]

I correct some typos, please read again...

Hello,

I have to be more precise...

I said that the TMREW from Omnithread
is not good cause the time of the TMREW's  reader section must be bigger than the time to execute those CASes on a global variable with cache misses,
more than that since the TMREW is using
a CAS on a global variable on the reader side also, hence it generates too much cahe-coherence traffic under contention , so this why TMREW from Omnithread is not scalable. But my scalable RWLock is scalable and it minimizes efficiently the cache-coherence traffic.


You can download my scalable RWLock from:

http://pages.videotron.com/aminer/



Thank you,
Amine Moulay Ramdane.

[zamtmn]

 
it's not funny and it is necessary to clean and ban

[aminer]

Hello again,

Other than that i have noticed that the OmniThread library(from htttp://otl.17slon.com/ ) is using a lockfree queue , so it not FIFO fair , hence it's also not starvation-free, and i think it's not scalable and it's bad, cause the Omnithread's lockfree queue generates  too much cache-coherence traffic under contention, this is so bad.

This is why i have implemented a concurrent FIFO queue that minimizes efficiently the cache-coherence traffic and that is FIFO fair and that is more energy efficient on the pop() size: it will
not spin-wait when there is no items in the queue, and i have used my concurrent FIFO queue inside my Threadpool engine.


Thank you,
Amine Moulay Ramdane.

[zamtmn]

how much evolved spam robots. they want to be like programmers

[aminer]

Hello all,


I have come to an interresting subject, i have implemented
ParallelVarFiler, it's a parallel hashtable that can be saved automaticly or manually to hardisk files.. so as you have
noticed you can save manually the parallel hashtable to the harddisk
by using the SaveToStream() or SaveToString() or SaveToFile() methods,
but to call those methods it's mandatory that you pass a file name
to the constructor, if you don't pass a file name to the constructor that means you are using an in-memory parallel hashtable, so if you
are using an in-memory parallel hashtable you have to stop your threads before calling  SaveToStream() or SaveToString() or SaveToFile() methods, and restart after that your threads, but
if you are saving automaticly your parallel hashtable to the hardisk
by passing a filename to the constructor, then you can safetly call the SaveToStream() or SaveToString() or SaveToFile() methods even if
you are using mutiple threads.


Hope you have understood the ideas behind my ParallelVarFiler.


You can download my ParallelVarFiler from:

http://pages.videotron.com/aminer/


Thank you,
Amine Moulay Ramdane.

[aminer]

I wrote:
>but to call those methods it's mandatory >that you pass a file name
>to the constructor

I mean when you are running mutliple threads.


Thank you,
Amine Moulay Ramdane.

[JanRoza]

aminer,

Is it very hard for you to just shut up and stop spamming this forum with your endless ranting?
You boring and irritating us with your monologues.

[aminer]

Hello,

I come to an interresting subject, i want to talk about the
scalability on multicores of the parallel hashtables and
databases engines... i am wondering how can you say that a parallel
hashtable can scale on multicores if the random key acesses are 
cache unfreiendly and the parallel hashtable is memory bound ? i think
since the parallel hashtable is memory bound and the random accesses are cache unfreindly that means that the parallel hashtable will not scale on multicores, when you are testing your parallel hashtable, you have also to test the cache misses on the data, cause i think that the data is part of the hashtable application, i mean the parallel hashtable concept is highly interconnected with the way that the data is accessed randomly in a cache unfreindly manner and the  hashtables concept is is highly interconnected with the way the hashtable data is accessed in memory bound manner, so this why i say that parallel hashtables are not scalable on multicores, it's the same for database engines , since they are disk bound, so they are not scalable, and even if they are in-memory databases they will be memory bound and cache unfreindly, so databases are not scable either on mutilcores.



Thank you,
Amine Moulay Ramndane

[aminer]

I correct some typos...

Hello,

I come to an interresting subject, i want to talk about the
scalability on multicores of the parallel hashtables and
databases engines... i am wondering how can you say that a parallel
hashtable can scale on multicores if the random key acesses are
cache unfreiendly and the parallel hashtable is memory bound ? i think
since the parallel hashtable is memory bound and the random accesses are cache unfriendly that means that the parallel hashtable will not scale on multicores, when you are testing your parallel hashtable, you have also to test the cache misses on the data, cause i think that the data is part of the hashtable application, i mean the parallel hashtable concept is highly interconnected with the way that the data is accessed randomly in a cache unfriendly manner and the  hashtables concept is  highly interconnected with the way the hashtable data is accessed in a memory bound manner, so this is why i say that parallel hashtables are not scalable on multicores, it's the same for database engines , since they are disk bound, so they are not scalable, and even if they are in-memory databases they will be memory bound and cache unfreindly, so databases are not scalable either on mutilcores.



Thank you,
Amine Moulay Ramndane

[aminer]

Hello,

We have to be smart please , i have come to an interresting subject...

If you look at those scalable FIFO queues...

http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=5&ved=0CFsQFjAE&url=http%3A%2F%2Fwww.cosy.sbg.ac.at%2Fresearch%2Ftr%2F2012-04_Kirsch_Lippautz_Payer.pdf&ei=FaBlUs-qIbTc4APonIHADQ&usg=AFQjCNHAhL-eCnvuCuNg8MSFhAfkRM73Ng&sig2=uy9PIKocjN9eXPAQQN8QlQ


http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=6&ved=0CGYQFjAF&url=http%3A%2F%2Fwww.cs.tau.ac.il%2F~shanir%2Fnir-pubs-web%2FPapers%2FSPAA2005.pdf&ei=FaBlUs-qIbTc4APonIHADQ&usg=AFQjCNGxX7ZAmgALfgRyL8WaDKjiYyw2uA&sig2=xAGjnkLOQf5bMhS4I2Wfbg


and this one:

https://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CDgQFjAB&url=https%3A%2F%2Frepository.ist.ac.at%2F124%2F1%2Fmain-spaa2013.pdf&ei=g6BlUqj7MJfi4AOp1oDYCg&usg=AFQjCNH9eKu7ZeXpaoxUhvTk7aW-akWrVg&sig2=lPv-uvllOBClf0xZjBSExA



Researchers will say that those FIFO queues are scalable, but
researchers are only testing pushes() and pops(), they are not testing
the cache misses on the data inside the FIFO queue, i mean we can
not look at the scalability ON MULTICORES from only the viewpoint of  the FIFO queue algotihms and appllication, because the FIFO queue concept is higly interconnected with how the data inside the FIFO queue is served in a memory bound manner
and in a cache-unfrienly manner, cause i think that the producers
threads will write the datas on records or objects and push
them on the FIFO queue, and the producers will have to have cache misses
on those datas , and the producers will access the data inside the
FIFO queue in a memory bound manner, so this is why i say that
those FIFO queues above are not scalable in pratice.

Hope you have understood what i mean.



Thank you,
Amine Moulay Ramdane.






 

[aminer]

Hello,

I have come to an interresting subject, i have tested
my Parallel archiver that is using parallel compression
and it is scaling but not perfectly, so why database
engines and hahstables are not scalable, since they are disk bound
or/and memory bound and your Parallel archiver is scaling but
not linearly and perfectly ? cause you have to understand
that parallel compression can be expensive, it's the compression
that is expensive and that can scale on multicores, but the memory data
tranfers are not scalable, it's the same for game engines
they will not gain on scalabilty on memory transfers , but
on rendering or compression or something like that that scales
on multicores, mathematic algorithms that dont use big memory data
transfers can scale also on multicores, but database engines
and hashtables are memory bound or/and disk bound etc. so they
are not scalable on multicores.


Hope you have understood my ideas...


Thank you,
Amine Moulay Ramdane.


 
.

[aminer]

Hello,

We have to be smart please, can you say Parallel compression
is only the compression excluding the data transfers in the parallel compression that is not scalable ?

Can you say a FIFO queue  is only the push and pop excuding the data tranfers indide the queue is not scalable ?


I think no, the data transfer that is not scalable is part of
the parallel compression and the FIFO queue.


This is why i have said that those FIFO queue that we call scalable
are in fact not scalable.


Thank you,
Amine Moulay Ramdane.

[aminer]

I correct some typos...

Hello,

We have to be smart please, can you say Parallel compression
is only the compression excluding the memory data transfers in the parallel compression that is not scalable ?

Can you say a FIFO queue  is only the push and pop excluding the memory data tranfers inside the queue that is not scalable ?


I think no, the memory data transfer that is not scalable is part of
the parallel compression and the FIFO queue.


This is why i have said that those FIFO queues that we call scalable
are in fact not scalable.


Thank you,
Amine Moulay Ramdane.