* * *

Author Topic: My following projects were updated or added  (Read 2326 times)

Sky68

  • New member
  • *
  • Posts: 29
Re: My following projects were updated or added
« Reply #15 on: July 04, 2018, 05:20:04 pm »
Hello,


Universal Scalability Law for Delphi and FreePascal for Windows and Linux version 3.2 is here..

Author: Amine Moulay Ramdane.

Description:

This program analyzes system performance data with the  Universal Scalability Law, and it compiles with Delphi XE  versions and FreePascal.

You have to supply the performance data as a csv file format, please take a look at the supplied csv file called "data.csv", the first line of the names of the columns of the csv file must be commented by "#" character, the names of the columns are mandatory.

Just compile the usl.pas program and run it by executing it
on the command prompt like this: usl data.csv

The Universal Scalability Law (USL) was developed by Dr. Neil J. Gunther. It can be used to analyze system performance data in order to learn more about the scalability limitations of the
system.

Details are presented in the book *Guerrilla Capacity Planning*.

Authors of Universal Scalability Law website:

http://www.perfdynamics.com/

Please take a look at the source code in the zip file:

When you compile the usl.pas , please type this at  the command prompt:  usl data.csv

Here is the output of my program:

Peak number is: 449.188
Predicted scalability peak is: 18.434
Coefficient of determination R-squared is: 0.995

The peak number is the peak number of cores (look inside the
csv file) that will give the Predicted scalability peak that is:
18.434X

I have included also usl_graph.exe and its source code, us_graph.exe will draw a graph of the predicted scalability of the USL law, please type at the commmand prompt:

usl_graph data.csv -g 5 449 -nlr

You have to give two parameters to the -g option, this will draw a graph with a step=d(x)=5 between two successive data points, and it will draw a graph up to the peak number that is x=449

You can save after that your graph to the clipboard and open
the paint program on windows and save it after that. The -nlr option means that the problem will be solved with the mathematical nonlinear regression using the simplex method as a minimization, if you don't specify -nlr, the problem will be
solved by default by the mathematical polynomial regression.

I have used a polynomial regression and i have done other approximations to find the predicted scalability peak when the derivative must equal an approximation of 0 and this when the
USL coefficient beta equal 0. This is all about mathematics.

You have three options:

You can type at the command prompt: usl data.csv -p 20
the -p option will give you the scalability for the data point 20
and you can type at the command prompt: usl data.csv -d 0.2 10
the -d option will give you the derivative of the USL equation at delta(y)/delta(x)=0.2 (it must be between 0 and 1) with a step delta(x)=10 that will output a number and derivative of a secant or a derivative delta(y)/delta(x) to better optimize the criterion of the cost for a better QoS.

And you can type at the command prompt: usl data.csv -nlr
the -nlr option means that the problem will be solved with the mathematical nonlinear regression using the simplex method as a minimization, if you don't specify -nlr, the problem will be solved by default by the mathematical polynomial regression.

I will a little bit explain my USL program...

If you have took a look at this link:

https://cran.r-project.org/web/packages/usl/vignettes/usl.pdf

You will notice that the performance data for the raytracer in the link above is the same as the performance data inside the data.csv file inside my zip file of my USL software..

And as you have noticed in the link above the peak scalability
number is at: 449 processors.

So if you run my program against this same performance data
like this at the command prompt:

usl data.csv

So the output is of my program is:

--

Peak number is: 449.188
Predicted scalability peak is: 18.434
Coefficient of determination R-squared is: 0.995

--

So as you have noticed that the peak number that is the peak number of processors is: 449.188

this is the same result as the link above.

So my program is working correctly.

But this is not the end of the story..

You have to optimize the criterion of the cost for a better QoS,
and for this i have supplied you with a second option called -d that you have to run for that, so you have to type at the command prompt:

usl data.csv -d 0.3 0.1

the 0.3 is the slope of the secant with a step 0.1, so since the step is 0.1 so this will approximate a derivative of the USL equation that equal 0.3, so here is the output of my program when you run it with -d 0.3 0.1:

--

Peak number is: 449.188
Predicted scalability peak is: 18.434
Coefficient of determination R-squared is: 0.995
The derivative of the USL equation at delta(y)/delta(x)=0.300
with a step delta(x)=0.100, gives a number and derivative of
a secant or a derivative delta(y)/delta(x) of: 16.600 and 0.300

--

So as you have noticed that a good approximation for the derivative of the USL equation will arrive at the 16.600 cores and this gives also a derivative of the secant that approximate the derivative of the USL equation.

So to optimize more the criterion of the cost for a better QoS, you have to choose a good delta(y)/delta(x) to optimize the criterion of the cost of your system and you have to balance better between the performance and the cost.

I have tested more my USL for Delphi and FreePascal and it is
working perfectly. But to make it work best with multicores
with the polynomial regression solver , you have to choose
the first column of the number of cores of the csv to: 1,2,4,8,16 without going up to 32 and it will work ok, don't choose 1,2,3,4,5 because this can make the polynomial regression solver to fail to solve the problem, but with the efficient nonlinear regression solver, you can make the first column of the number of cores of the csv to: 1,2,3,4,5 cores, and it will work correctly and solve the problem, because the nonlinear regression solver is more efficient.

How can you be sure that my USL program for Delphi and FreePascal works correctly ?

Please take a look at this link:

https://cran.r-project.org/web/packages/usl/vignettes/usl.pdf

Notice the raytracer performance data, when they have
analysed it, it gives a peak scalability at: 449

So try to run my program inside the zip against the the same
raytracer performance data that you will find inside the
data.csv file inside the zip, and this will give the same peak
scalability at: 449.

So as you have noticed, my program is working for this performance data of the raytracer, so i think that you can be
confident with my program.

I have included a 32 bit and 64 bit windows executables called
usl.exe and usl_graph.exe inside the zip, please read the  readme file to know how to use it, it is a very powerful tool.

Here is an important and free book about Universal Scalability
Law (USL), read it to understand more what USL is all about, you can download it from this link:

https://www.vividcortex.com/resources/universal-scalability-law/


You can download Universal Scalability Law for Delphi and FreePascal from:

https://sites.google.com/site/scalable68/universal-scalability-law-for-delphi-and-freepascal

Language: FPC Pascal v2.2.0+ / Delphi 5+: http://www.freepascal.org/

Required FPC switches: -O3 -Sd

-Sd for delphi mode....

Required Delphi switches: -DMSWINDOWS -$H+ -DDelphi

Required Delphi XE-XE7 switch: -$H+ -DXE

For Delphi use -DDelphi

- Platform: Win32 ,Win64,Linux,OSX


Thank you,
Amine Moulay Ramdane.

Thaddy

  • Hero Member
  • *****
  • Posts: 6362
Re: My following projects were updated or added
« Reply #16 on: July 04, 2018, 05:38:46 pm »
@Amine
Your code is good, but you run the risk of SPAM.
My suggestion is to refer to your library repository and announce there is an update.
Still: good work! and thanks for sharing!
I am enjoining wine, not whine....

Sky68

  • New member
  • *
  • Posts: 29
Re: My following projects were updated or added
« Reply #17 on: July 09, 2018, 02:11:02 am »
Hello...

As you have noticed i am an "inventor" of many scalable algorithms and there implementations, and as you have noticed Embarcadero company that actually sells its Delphi product has pushed to adopt the ARC memory manager despite it being not the right choice for highly parallel applications, so i have decided to invent a "scalable" reference counting with efficient support for weak references that is suited also for "highly" parallel applications and i have implemented it in Delphi and FreePascal, and you will not find it on C++ or Rust or Delphi, here is my "invention" and read about it and download it from bellow:

Scalable reference counting with efficient support for weak references version 1.12

Author: Amine Moulay Ramdane

Description:

I have enhanced my scalable algorithm and now it is much powerful, now my scalable algorithm implementation works also as a "scalable" counter that supports both "increment" and "decrement" using two scalable counting networks, please take a look at my new scalable algorithm implementation inside the source code..

This is my scalable reference counting with efficient support for weak references, and since problems that cannot be solved without weak references are rare, so this library does scale very well, this scalable reference counting is implemented using scalable counting networks that eliminate completely false sharing , so it is fully scalable on multicore processors and manycore processors and this scalable algorithm is optimized, and this library does work on both Windows and Linux (x86), and it is easy to port to Mac OS X.

I have modified my scalable algorithm, now as you will notice i am not using decrement with support for antitokens in the balancers of the scalable counting networks, i am only using an "increment", please look at my new scalable algorithm inside the zip file, i think it is working correctly. Also notice that the returned value of _Release() method will be valid if it is equal to 0.

I have optimized it more, now i am using only tokens and no antitokens in the balancers of the scalable counting networks, so i am only supporting increment, not decrement, so you have to be smart to invent it correctly, this is what i have done, so look at the AMInterfacedObject.pas file inside my zip file, you will notice that it uses counting_network_next_value() function, counting_network_next_value() increments the scalable counting network by 1, the _AddRef() method is simple, it increment by 1 to increment the reference to the object, but look inside the _Release() method it calls counting_network_next_value() three times, and my invention is calling counting_network_next_value(cn1) first inside the _Release() method to be able to make my scalable algorithm works, so just debug it more and you will notice that my scalable algorithm is smart and it is working correctly, i have debugged it and i think it is working correctly.

I have to prove my scalable reference counting algorithm, like with mathematical proof, so i will use logic to prove like in PhD papers:

You will find the code of my scalable reference counting inside AMInterfacedObject.pas inside the zip file here:

If you look inside the code there is two methods, _AddRef() and _Release() methods, i am using two scalable counting networks,
think about them like counters, so in the _AddRef() method i am
executing the following:

v1 := counting_network_next_value(cn1);

cn1 is the scalable counting network, and counting_network_next_value() is a function that increment the scalable counting network by 1.

In the _Release() method i am executing the following:

v2 := counting_network_next_value(cn1);
v1 := counting_network_next_value(cn2);
v1 := counting_network_next_value(cn2);

So my scalable algorithm is "smart", because the logical proof is
that i am calling counting_network_next_value(cn1) first in the
above, so this allows my scalable algorithm to work correctly,
because we are advancing cn1 by 1 to obtain the value of cn1,
so the other threads are advancing also cn1 by one inside
_Release() , it is the last thread that is advancing cn1 by 1 that will make the reference counter equal to 0 , and _AddRef() method is the same and it is easy to reason about, so this scalable algorithm is working. Please look more carefully at my algorithm and you will notice that it is working as i have just logically proved it.

Please read also the following to understand better:

Here is the parameters of the constructor:

First parameter is: The width of the scalable counting networks that permits my scalable refererence counting algorithm to be scalable, this parameter must be 1 to 31, it is now at 4 , this is the power, so it is equal to 2 power 4 , that means 2^4=16, and you have to pass this counting networks width to the n of following formula:

(n*log(n)*(1+log(n)))/4

The log of the formula is in base 2

This formula gives the number of gates of the scalable counting networks, and if we replace n by 16, this will equal 80 gates, that means you can scale the scalable counting networks to 80 cores, and beyond 80 cores you will start to have contention.

Second parameter is: a boolean that tells if reference counting is used or not, it is by default to true, that means that reference counting is used.

About the weak references support: the Weak<T> type supports assignment from and to T and makes it usable as if you had a variable of T. It has the IsAlive property to check if the reference is still valid and not a dangling pointer. The Target property can be used if you want access to members of the reference.

Note: the use of the IsAlive property on our weak reference, this tells us whether the referenced object is still available, and provides a safe way to get a concrete reference to the parent.

I have ported efficient weak references support to Linux by implementing efficient code hooking, look at my DSharp.Core.Detour.pas file for Linux that i have written to see how i have implemented it in the Linux library. Please look at the example.dpr and test.pas demos to see how weak references work etc.

Call _AddRef() and _Release() methods to manually increment or decrement the number of references to the object.

Weak references support is done by hooking the TObject.FreeInstance method so every object destruction is noticed and if a weak reference for that object exists it gets removed from the internal dictionary where all weak references are stored. While it works I am aware that this is hacky approach and it might not work if someone overrides the FreeInstance method and does not call inherited.


You can download it from:

https://sites.google.com/site/scalable68/scalable-reference-counting-with-efficient-support-for-weak-references

- Platform: Windows and Linux(x86)

Language: FPC Pascal v3.1.x+ / Delphi 2007+:

http://www.freepascal.org/

Required FPC switches: -O3 -Sd

-Sd for delphi mode....

Required Delphi switches: -$H+ -DDelphi

For Delphi XE versions and Delphi Tokyo use the -DXE switch

The defines options inside defines.inc are:

{$DEFINE CPU32} for 32 bit systems

{$DEFINE CPU64} for 64 bit systems



Thank you,
Amine Moulay Ramdane.



Sky68

  • New member
  • *
  • Posts: 29
Re: My following projects were updated or added
« Reply #18 on: July 09, 2018, 11:18:33 pm »
Hello...

Read this:

My Scalable reference counting with efficient support for weak references was updated to version 1.2

My new scalable algorithm was enhanced to ensure that it is thread-safe, i have also benchmarked it and it is really amazing ! because it is really fast and it is "scalable"(on NUMA systems and multicores).

As you have noticed Embarcadero company that actually sells its Delphi product has pushed to adopt the ARC memory manager despite it being not the right choice for highly parallel applications, so i have decided to invent a "scalable" reference counting with efficient support for weak references that is suited also for "highly" parallel applications and i have implemented it in Delphi and FreePascal, and you will not find it on C++ or Rust or Delphi, here is my "invention" and read about it and download it from bellow:

Scalable reference counting with efficient support for weak references version 1.2

Author: Amine Moulay Ramdane

Description:

I have enhanced my scalable algorithm and now it is much powerful, now my scalable algorithm implementation works also as a "scalable" counter that supports both "increment" and "decrement" using two scalable counting networks, please take a look at my new scalable algorithm implementation inside the source code..

This is my scalable reference counting with efficient support for weak references, and since problems that cannot be solved without weak references are rare, so this library does scale very well, this scalable reference counting is implemented using scalable counting networks that eliminate completely false sharing , so it is fully scalable on multicore processors and manycore processors and this scalable algorithm is optimized, and this library does work on both Windows and Linux (x86), and it is easy to port to Mac OS X.

I have modified my scalable algorithm, now as you will notice i am not using decrement with support for antitokens in the balancers of the scalable counting networks, i am only using an "increment", please look at my new scalable algorithm inside the zip file, i think it is working correctly. Also notice that the returned value of _Release() method will be valid if it is equal to 0.

I have optimized it more, now i am using only tokens and no antitokens in the balancers of the scalable counting networks, so i am only supporting increment, not decrement, so you have to be smart to invent it correctly, this is what i have done, so look at the AMInterfacedObject.pas file inside my zip file, you will notice that it uses counting_network_next_value() function, counting_network_next_value() increments the scalable counting network by 1, the _AddRef() method is simple, it increment by 1 to increment the reference to the object, but look inside the _Release() method it calls counting_network_next_value() three times, and my invention is calling counting_network_next_value(cn1) first inside the _Release() method to be able to make my scalable algorithm works, so just debug it more and you will notice that my scalable algorithm is smart and it is working correctly, i have debugged it and i think it is working correctly.

I have to prove my scalable reference counting algorithm, like with mathematical proof, so i will use logic to prove like in PhD papers:

You will find the code of my scalable reference counting inside AMInterfacedObject.pas inside the zip file here:

If you look inside the code there is two methods, _AddRef() and _Release() methods, i am using two scalable counting networks,
think about them like counters, so in the _AddRef() method i am
executing the following:

v1 := counting_network_next_value(cn1);

cn1 is the scalable counting network, and counting_network_next_value() is a function that increment the scalable counting network by 1.

In the _Release() method i am executing the following:

v2 := counting_network_next_value(cn1);
v1 := counting_network_next_value(cn2);
v1 := counting_network_next_value(cn2);

So my scalable algorithm is "smart", because the logical proof is
that i am calling counting_network_next_value(cn1) first in the
above, so this allows my scalable algorithm to work correctly,
because we are advancing cn1 by 1 to obtain the value of cn1,
so the other threads are advancing also cn1 by one inside
_Release() , it is the last thread that is advancing cn1 by 1 that will make the reference counter equal to 0 , and _AddRef() method is the same and it is easy to reason about, so this scalable algorithm is working. Please look more carefully at my algorithm and you will notice that it is working as i have just logically proved it.

Please read also the following to understand better:

Here is the parameters of the constructor:

First parameter is: The width of the scalable counting networks that permits my scalable refererence counting algorithm to be scalable, this parameter must be 1 to 31, it is now at 4 , this is the power, so it is equal to 2 power 4 , that means 2^4=16, and you have to pass this counting networks width to the n of following formula:

(n*log(n)*(1+log(n)))/4

The log of the formula is in base 2

This formula gives the number of gates of the scalable counting networks, and if we replace n by 16, this will equal 80 gates, that means you can scale the scalable counting networks to 80 cores, and beyond 80 cores you will start to have contention.

Second parameter is: a boolean that tells if reference counting is used or not, it is by default to true, that means that reference counting is used.

About the weak references support: the Weak<T> type supports assignment from and to T and makes it usable as if you had a variable of T. It has the IsAlive property to check if the reference is still valid and not a dangling pointer. The Target property can be used if you want access to members of the reference.

Note: the use of the IsAlive property on our weak reference, this tells us whether the referenced object is still available, and provides a safe way to get a concrete reference to the parent.

I have ported efficient weak references support to Linux by implementing efficient code hooking, look at my DSharp.Core.Detour.pas file for Linux that i have written to see how i have implemented it in the Linux library. Please look at the example.dpr and test.pas demos to see how weak references work etc.

Call _AddRef() and _Release() methods to manually increment or decrement the number of references to the object.

Weak references support is done by hooking the TObject.FreeInstance method so every object destruction is noticed and if a weak reference for that object exists it gets removed from the internal dictionary where all weak references are stored. While it works I am aware that this is hacky approach and it might not work if someone overrides the FreeInstance method and does not call inherited.


You can download it from:

https://sites.google.com/site/scalable68/scalable-reference-counting-with-efficient-support-for-weak-references

- Platform: Windows and Linux(x86)

Language: FPC Pascal v3.1.x+ / Delphi 2007+:

http://www.freepascal.org/

Required FPC switches: -O3 -Sd

-Sd for delphi mode....

Required Delphi switches: -$H+ -DDelphi

For Delphi XE versions and Delphi Tokyo use the -DXE switch

The defines options inside defines.inc are:

{$DEFINE CPU32} for 32 bit systems

{$DEFINE CPU64} for 64 bit systems



Thank you,
Amine Moulay Ramdane.


zamtmn

  • Sr. Member
  • ****
  • Posts: 327
Re: My following projects were updated or added
« Reply #19 on: July 09, 2018, 11:38:01 pm »
this is spam. I'm surprised at the patience of the moderators

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus