Lazarus

Programming => Packages and Libraries => Topic started by: Xor-el on August 03, 2019, 11:34:48 am

Title: NumCPULib4Pascal
Post by: Xor-el on August 03, 2019, 11:34:48 am
NumCPULib4Pascal is a Library that enables you to Count the number of CPUs on the current machine.
This Library aims to provide support for those Oses that are not supported by System.CPUCount ( or those in which System.CPUCount defaults to 1).

Code: Pascal  [Select]
  1. uses
  2.   NumCPULib;
  3.  
  4. var
  5.   lcc, pcc: Int32;
  6. begin
  7. // count logical cpus (aka logical processors)
  8.  lcc := TNumCPULib.GetLogicalCPUCount();
  9. // count physical cpus (aka cores)
  10.  pcc := TNumCPULib.GetPhysicalCPUCount();
  11. end;
  12.  

Supported Oses

https://github.com/Xor-el/NumCPULib4Pascal (https://github.com/Xor-el/NumCPULib4Pascal)
Title: Re: NumCPULib4Pascal
Post by: totya on August 03, 2019, 11:51:15 am
NumCPULib4Pascal is a Library that enables you to Count the number of CPUs on the current machine.
This Library aims to provide support for those Oses that are not supported by System.CPUCount ( or those in which System.CPUCount

Hi, thank you for this new library!

The UsageSamples.lpr (in FreePascal.Samples lol?) works without any problem on Win7 x64, compiled either x86 or x64, and show my proc core count: 2.

Ony few unimportant messages come in under compile (as usually):
Quote
Compile Project, Target: UsageSamples.exe: Success, Hints: 3
NumCPULib.pas(367,24) Hint: Local variable "LSystemAffinityMask" does not seem to be initialized
NumCPULib.pas(366,68) Hint: Local variable "LProcessAffinityMask" does not seem to be initialized
NumCPULib.pas(382,30) Hint: Local variable "LSystemInfo" does not seem to be initialized
501 lines compiled, 0.2 sec, 72560 bytes code, 4196 bytes data
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 03, 2019, 12:01:29 pm
@totya, Thanks, have fixed the compiler warnings with a commit just now.
Title: Re: NumCPULib4Pascal
Post by: totya on August 03, 2019, 12:08:35 pm
No warning messages come in with rev3, when I compile the UsageSamples.lpr (lazarus project file) under FreePascal.Samples directory, thanks!
Title: Re: NumCPULib4Pascal
Post by: marcov on August 03, 2019, 01:32:22 pm
Hmm, it shows 8 on my 4 core (and probably will show 12 on my 6 core) due to hyperthreading.  The logical count number is not very useful, usually the number of worker threads to spawn is based on physical count (and in case of hyperthreading a slightly higher percentage on top of that)

is there some way to get the physical count?

Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 03, 2019, 02:08:21 pm
Hmm, it shows 8 on my 4 core (and probably will show 12 on my 6 core) due to hyperthreading.  The logical count number is not very useful, usually the number of worker threads to spawn is based on physical count (and in case of hyperthreading a slightly higher percentage on top of that)

is there some way to get the physical count?

for now, only Logical CPU Count is supported by this library.
Physical CPU Count is possible but is supported only on Linux, mac OS and Windows platforms (at least from the little research I have made).
Title: Re: NumCPULib4Pascal
Post by: totya on August 03, 2019, 02:22:40 pm
is there some way to get the physical count?

I thinking on it about few years ago... the REAL CORE (OS usable) detect method is very simple on any OS.. Just open and execute a new thread with any time-measurable operation. About: if the exists thread(s) execution time significantly decreased or the new thread speed is significantly ower than previous thread speeds, then the new thread can't assign to new cpu core, so you get the real cpu core count.

Certainly, when run this measure, no other app running with significant cpu usage... but to tell the truth, when your app is started, you need to know, what is the FREE and usable CPU core count, isn't it? ;)
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 12:31:14 pm
v1.2 Update

added support for physical CPU Count.

Supported Oses for Physical CPU Count are same as that of Logical CPU Count with the exception of the BSD family (FreeBSD, NetBSD, OpenBSD, DragonFlyBSD).

Linux boxes are supported but I have not been able to test on RPI. if anyone can do that, I will be grateful.

also @marcov, can you please confirm that the Physical CPU Count are corrected reported on your Boxes with this latest update?
Title: Re: NumCPULib4Pascal
Post by: Thaddy on August 07, 2019, 01:31:22 pm
I will test the RPi and will report back.
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 01:40:03 pm
I will test the RPi and will report back.

Thanks. :)
Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 05:12:40 pm
Hi!

There was an example how to count CPUs  in the wiki, now it is an empty page.  It was at
http://wiki.freepascal.org/Example_of_multi-threaded_application (http://wiki.freepascal.org/Example_of_multi-threaded_application)

It could count only logic CPUs, not physical. It worked on Linux, Darwin, Windows, Solaris and FreeBSD as I remember.

If it helps I can start a search in the deepth of my hardisk - must be somewhere.

Winni
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 05:39:28 pm
Hi!

There was an example how to count CPUs  in the wiki, now it is an empty page.  It was at
http://wiki.freepascal.org/Example_of_multi-threaded_application (http://wiki.freepascal.org/Example_of_multi-threaded_application)

It could count only logic CPUs, not physical. It worked on Linux, Darwin, Windows, Solaris and FreeBSD as I remember.

If it helps I can start a search in the deepth of my hardisk - must be somewhere.

Winni

@Winni, I really don't know about that but my library aim to support more than just logical CPUs count, it supports Physical CPUs count too.
Also it is Delphi and FPC compatible too.
Title: Re: NumCPULib4Pascal
Post by: totya on August 07, 2019, 05:39:43 pm
Windows 7 result:

Quote
Logical CPU Count is 2
Physical CPU Count is 2

It's okay. But to tell the truth, I have one CPU with two cores. ;)
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 05:40:34 pm
Windows 7 result:

Quote
Logical CPU Count is 2
Physical CPU Count is 2

It's okay. But to tell the truth, I have one CPU with two cores. ;)

Was this by any chance run in a VM?
By the way, this is strange, it works beautifully for me on my windows box reporting 2 physical CPUs and 4 logical CPUs.
Title: Re: NumCPULib4Pascal
Post by: Thaddy on August 07, 2019, 05:47:02 pm
Works perfectly fine on the Raspberry Pi (all versions)
Used this code, because I could not stand the superfluous format() use  ;D ;D
Code: Pascal  [Select]
  1. program cpulibtest;
  2. uses
  3.   numcpulib;
  4. begin
  5.   writeLn('Logical CPU Count is ', TNumCPULib.GetLogicalCPUCount);
  6.   writeLn('Physical CPU Count is ', TNumCPULib.GetPhysicalCPUCount);
  7. end.

Outputs on RPi4 (and 2 and 3):
Code: Bash  [Select]
  1. Logical CPU Count is 4
  2. Physical CPU Count is 4

Raspberry Pi 1 B and zero/w:
Code: Bash  [Select]
  1. Logical CPU Count is 1
  2. Physical CPU Count is 1
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 05:48:04 pm
Works perfectly fine on the Raspberry Pi (all versions)
Used this code, because I could not stand the superfluous format() use  ;D ;D
Code: Pascal  [Select]
  1. program cpulibtest;
  2. uses
  3.   numcpulib;
  4. begin
  5.     WriteLn('Logical CPU Count is ', TNumCPULib.GetLogicalCPUCount);
  6.     WriteLn('Physical CPU Count is ', TNumCPULib.GetPhysicalCPUCount);
  7. end.

Outputs:
Code: Bash  [Select]
  1. Logical CPU Count is 4
  2. Physical CPU Count is 4

Lol, thanks for testing.
Title: Re: NumCPULib4Pascal
Post by: totya on August 07, 2019, 05:49:17 pm
Was this by any chance run in a VM?

My english is very weak, so my answer what I understood, so I don't use any virtual machine now, this sample run on my "clean" windows 7 x 64 OS.
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 05:54:39 pm
Was this by any chance run in a VM?

My english is very weak, so my answer what I understood, so I don't use any virtual machine now, this sample run on my "clean" windows 7 x 64 OS.

And you are sure your PC has just one physical CPU?
Title: Re: NumCPULib4Pascal
Post by: totya on August 07, 2019, 06:06:30 pm
And you are sure your PC has just one physical CPU?

I repair the computers in my workplace becuase the local IT cant fix anything, so what do you think, can I know how many cpu sleep in my mainboard? :)

Edit.: I added a screenshot of motheboard page from the CPU-Z app too ;)
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 06:21:08 pm
And you are sure your PC has just one physical CPU?

I repair the computers in my workplace becuase the local IT cant fix anything, so what do you think, can I know how many cpu sleep in my mainboard? :)

sorry if my question seemed out of place but your PC has 2 Cores which CPU-Z and My Library agrees on.  :)

Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 07:10:07 pm
Hi!

Hardware: AMD Ryzen 5 2400 g (4 cores, 8 threads)
OS: Suse Tumbleweed

reports:

Logical CPU Count is 8
Physical CPU Count is 8

What kind of count is going on?

Winni





Title: Re: NumCPULib4Pascal
Post by: TraumTaenzerDieter on August 07, 2019, 08:06:54 pm
Hi,
my hardware: AMD Ryzen 5 1500x (4 Cores, 8 Threads),
my OS: Win7 64

report says:
physical 4, logical 8,
so everything is okay!
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 08:19:10 pm
Hi!

Hardware: AMD Ryzen 5 2400 g (4 cores, 8 threads)
OS: Suse Tumbleweed

reports:

Logical CPU Count is 8
Physical CPU Count is 8

What kind of count is going on?

Winni

Hi Winni, is this a Virtual Machine?
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 08:21:39 pm
Hi,
my hardware: AMD Ryzen 5 1500x (4 Cores, 8 Threads),
my OS: Win7 64

report says:
physical 4, logical 8,
so everything is okay!

Thanks for testing.
Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 08:50:24 pm
@Xor-el

No, this is native Suse Linux!

Winni
Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 09:04:33 pm
@Xor-el

The AMD processor page says definitly: 4 cores, 8 threads

The KDE Info Center reports 8 cores!

HardInfo reports 8 processors but doesn't say nothing if physical or logical!

This is a land of confusion ....

Winni
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 09:24:47 pm
@Xor-el

The AMD processor page says definitly: 4 cores, 8 threads

The KDE Info Center reports 8 cores!

HardInfo reports 8 processors but doesn't say nothing if physical or logical!

This is a land of confusion ....

Winni

Confusion everywhere  :D :D :D :D :D :D :D
Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 10:25:55 pm
@Xor-el

In the case of linux it seems to be the easiest way to read /proc/cpuinfo. There is a very lot of information, but the two needed values are present:

......
processor       : 7
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 17
model name      : AMD Ryzen 5 2400G with Radeon Vega Graphics
stepping        : 0
microcode       : 0x810100b
cpu MHz         : 1458.000
cache size      : 512 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
.........

That seems to be a secure and easy way.

Winni
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 07, 2019, 10:30:42 pm
@Xor-el

In the case of linux it seems to be the easiest way to read /proc/cpuinfo. There is a very lot of information, but the two needed values are present:

......
processor       : 7
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 17
model name      : AMD Ryzen 5 2400G with Radeon Vega Graphics
stepping        : 0
microcode       : 0x810100b
cpu MHz         : 1458.000
cache size      : 512 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
.........

That seems to be a secure and easy way.

Winni

I do know but if you look carefully, you will notice there are about 8 occurences of cpu cores       : 4 in your output
Title: Re: NumCPULib4Pascal
Post by: winni on August 07, 2019, 10:54:58 pm
Yes the info in /proc/cpuinfo is a little bit complicated. In the moment I found only a german page which explains it:
https://linuxwiki.de/proc/cpuinfo (https://linuxwiki.de/proc/cpuinfo)

The main issues are:

For every logical CPU there is an info block in the file - in your case 8, in mine 8.

So there is redundand info in the blocks. But even if it 4 or 8 times appears - the info is: There are 4 cores. And the same with the siblings. So dont't be afraid that you got suddenly 32 cores!

Winni
Title: Re: NumCPULib4Pascal
Post by: ASBzone on August 08, 2019, 03:16:54 am
NumCPULib4Pascal is a Library that enables you to Count the number of CPUs on the current machine.
This Library aims to provide support for those Oses that are not supported by System.CPUCount ( or those in which System.CPUCount defaults to 1).

Awesome!  And thanks!   I was researching code for this very thing in June...
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 08, 2019, 12:12:47 pm
NumCPULib4Pascal is a Library that enables you to Count the number of CPUs on the current machine.
This Library aims to provide support for those Oses that are not supported by System.CPUCount ( or those in which System.CPUCount defaults to 1).

Awesome!  And thanks!   I was researching code for this very thing in June...

Glad I could help.  8)
Title: Re: NumCPULib4Pascal
Post by: Thaddy on August 08, 2019, 02:10:49 pm
@Xor-el
I have one remark:
You designed it with int32 and exceptions.
I would prefer it without exceptions and an unsigned type.
For some platforms that would require maybe a local variable of a signed type.
After all you have a fall-back to one, and it can't be processed at all of there is no processor  :D 8-)
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 08, 2019, 02:59:43 pm
@Xor-el
I have one remark:
You designed it with int32 and exceptions.
I would prefer it without exceptions and an unsigned type.
For some platforms that would require maybe a local variable of a signed type.
After all you have a fall-back to one, and it can't be processed at all of there is no processor  :D 8-)

thanks for the remark, will look into it. :)
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 08, 2019, 06:46:00 pm
@Xor-el
I have one remark:
You designed it with int32 and exceptions.
I would prefer it without exceptions and an unsigned type.
For some platforms that would require maybe a local variable of a signed type.
After all you have a fall-back to one, and it can't be processed at all of there is no processor  :D 8-)

@Thaddy, as promised
https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas (https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas)
Can you please confirm that I have implemented your remark so I can merge it to master?
Title: Re: NumCPULib4Pascal
Post by: Thaddy on August 08, 2019, 06:55:02 pm
need 24 hours for proper reply. Looks good.
Title: Re: NumCPULib4Pascal
Post by: winni on August 08, 2019, 08:12:42 pm
@Xor-el

Just tested the version https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas (https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas)

Yeah, now with info from /proc/cpuinfo everythings looks fine.

For my Ryzen  is reported

Logical CPU Count is 8
Physical CPU Count is 4

Have a nice day
Winni
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 08, 2019, 08:55:38 pm
@Xor-el

Just tested the version https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas (https://github.com/Xor-el/NumCPULib4Pascal/blob/travis/NumCPULib/src/NumCPULib.pas)

Yeah, now with info from /proc/cpuinfo everythings looks fine.

For my Ryzen  is reported

Logical CPU Count is 8
Physical CPU Count is 4

Have a nice day
Winni

@winni, yes I did make some minor changes as regards multiple sockets PC.
was not making a unique count before so duplicates were counted.

Thanks for testing though.
Title: Re: NumCPULib4Pascal
Post by: ASBzone on August 08, 2019, 10:16:28 pm

@winni, yes I did make some minor changes as regards multiple sockets PC.
was not making a unique count before so duplicates were counted.

Thanks for testing though.


Question, @Xor-el.   At present, do you plan to be able to distinguish a system with 2 sockets, each containing quad-core processors, with hyperthreading, vs a single 8-core CPU with hyperthreading?

Both would have 8 physical CPUs and 16 logical CPUs, but it would be nice to be able to distinguish two-socket or four-socket server system.
Title: Re: NumCPULib4Pascal
Post by: Xor-el on August 08, 2019, 10:44:51 pm

Question, @Xor-el.   At present, do you plan to be able to distinguish a system with 2 sockets, each containing quad-core processors, with hyperthreading, vs a single 8-core CPU with hyperthreading?

Both would have 8 physical CPUs and 16 logical CPUs, but it would be nice to be able to distinguish two-socket or four-socket server system.

@ASBzone, might be a research area for me in the future but for now nothing at the moment.  :)
Title: Re: NumCPULib4Pascal
Post by: ASBzone on August 10, 2019, 12:34:01 am
@ASBzone, might be a research area for me in the future but for now nothing at the moment.  :)


Fair enough.  Thanks for the work in any event.  It is much appreciated.  I recently spent a good part of June looking for code that would do just this, and had to put everything on hold.  But now, your unit saves the day.  Thank you kindly.