Hardware Random

[mercurhyo]

Hi everyone.

I wrote in 2 minutes this little function :

Code: Pascal  [Select][+][-]

1. {$mode ObjFPC}{$H+}
2. {$asmmode intel}
3.  
4. interface
5.  
6. uses
7.   Classes, SysUtils;
8.  
9. function getrand: word; // try to fill result with hardware generated random number
10.  
11. implementation
12.  
13. function getrand: word;
14. label
15.   noway;
16. var
17.   ok: boolean;
18. begin
19.   Result := $ffff;
20.   ok := False;
21.  
22.   asm
23.            XOR     RAX,RAX
24.            CPUID                       // initiates leafs in case of
25.            MOV     RAX,1               // cpuid standard function 1
26.            CPUID
27.            TEST    RCX,20000000H       // if bit 30 is 1
28.            JZ      noway
29.            MOV     [ok],True           // then your CPU is able to get hardware generated random numbers
30.            noway:
31.   end;
32.  
33.   if not ok then Exit;                 // if not able, getrand will always return $FFFF
34.  
35.   asm
36.            RDRAND  AX;                 // pick a random number
37.            MOV     [Result],AX         // return it to caller
38.   end;
39.  
40.   // TODO : improve the getrand function to return numbers between intervals of 8/16/32/64 bits integers
41.  
42. end;
43.  

so it takes the advantage of modern CPUs to make TRUE random numbers (hardware)

I am wondering why FPC do not use this capability if available, alongside its Mersenne Twister algo implementation...?

Thanks

[korba812]

I am wondering why FPC do not use this capability if available, alongside its Mersenne Twister algo implementation...?

Because your solution is platform specific?

[mercurhyo]

I am wondering why FPC do not use this capability if available, alongside its Mersenne Twister algo implementation...?

Because your solution is platform specific?

absolutely not

[wildfire]

I am wondering why FPC do not use this capability if available, alongside its Mersenne Twister algo implementation...?

Because your solution is platform specific?

absolutely not

Well that code won't run on any of my Pi's.

[Kays]

[…] so it takes the advantage of modern CPUs to make TRUE random numbers (hardware) […]

“True” is an overstatement, because rdRand is a DBRG, a deterministic random bit generator, that, however, uses a non‑deterministic entropy source. Arguably it may be of better quality, but it is not as truly random as, for example, radioactive decay can be.

[…] I am wondering why FPC do not use this capability if available, alongside its Mersenne Twister algo implementation...? […]

• There is a desire and expectation and that given the same randSeed, random reproduces the same sequence of random numbers. This cannot be achieved with rdRand.
• As far as I understand it is also a matter of performance: rdRand is designed for cryptographic applications in mind and as such frequently consults the built‑in entropy pool, overall slowing down operations. You want to use an MT if you need many random numbers, e. g. in statistical applications.
• Code needs to be maintained. Unless there is a compelling reason to include more code thus increasing the maintenance burden, you refrain from including more features.

_{PS: You need to check the carry flag, because rdRand may return zero in certain circumstances.}

[mercurhyo]

i do not undertand your point Kays.

So... FPK can not add new features because there is maintnance of 40 years old libs ... LOL huh they did it many times

to me, it would have been (since 2012-14) quiet simple to add a function for example named 'HardRand(limit: longint)' or a compiler directive like '$RNDGEN ON|OFF' modifying the behavior of std 'Random' call

[TRon]

So... FPK can not add new features because there is maintnance of 40 years old libs ... LOL huh they did it many times

to me, it would be quiet simple to add a function named 'HardRand(limit: longint)' or a compiler directive like '$RNDGEN ON|OFF' modifying the behavior of std random call

Ofc. but that is besides the point.

The random function works in a certain manner and is expected to work like that. Not adhering to that would introduce a lot of problems for people who relies on the (existing) behaviour.

Ofc you can debate whether or not Free Pascal should offer different implementations of the random function but they can be added like any other package (the wiki has plenty of different implementation thanks to f.e. Thaddy).

If you want the standard random function to act differently then simply 'override' the existing implementation (adding your custom implementation unit later in the chain)  and be done with it.

BTW: nothing that was said makes the shown implementation cross-platform compatible though 

[mercurhyo]

@TRon

we should debate what 'platform' means in 'crossplatform'


Does it rely on CPU models or on OSes types or both ? the term 'crossplatform' becomes more and more deep in fog.


my 1st impression was that 'crossplatform' rely on OSes but now I think this expression is much more complex and AMBIGUOUS

[TRon]

we should debate what 'platform' means in 'crossplatform'

We can ofc. but fwiw I was not referring to JITT translation (layer).

Does it rely on CPU models or on OSes types or both ? the term 'crossplatform' becomes more and more deep in fog.

Yes,  it is indeed.

my 1st impression was that 'crossplatform' rely on OSes but now I think this expression is much more complex and AMBIGUOUS

It is the CPU that determines if code is compatible. But you are correct in that JITT translation layers make it more difficult to distinguish between these kind of things.

For the (Free Pascal) compiler things are more simpler: it creates code for a certain CPU and OS combination (or embedded).

If you introduce assembler in your code you have to make sure it matches the CPU it is compiled for. The code as shown does not compile for arm, m68k, ppc CPU's to name a few. Note that M$ is pretty much full in on the arm things (for which they will fail, again, is my prediction)

[Kays]

[…] to me, it would have been (since 2012-14) quiet simple to add a function for example named 'HardRand(limit: longint)' or a compiler directive like '$RNDGEN ON|OFF' modifying the behavior of std 'Random' call

Well, if it so simple why don’t you file a merge request and the developers will incorporate it or turn it down for reasons they explain.

[Thaddy]

It is so simple that it has already been done over a couple of decades many times. It is a hexagon instead of a circle, And the suggestion is not portable. If supported the instruction is already used in almost all OS's by default.

It is, actually, a lovely kind of naivity.
We all started like that.

[Thaddy]

To check your random and to check if it is or approximates TRNG you can install rng-tools and run:

Code: Bash  [Select][+][-]

1. sudo cat /dev/random | rngtest -c 20000

You can use several runs. Fails are expected but should be around .2 - .5% or less if hardware entropy is added to the entropy pool. On modern Intel platforms and e.g. the Raspberry Pi this is already always the case.
Some cpu's also have a /dev/hwrnd device. There really is no need to call the rdrand/rdseed instructions from assembler. The OS will do that for you.
Note that /dev/random is blocking, while /dev/urandom is non-blocking.
On Windows it is similar to unixes: if hardware entropy is available it is used.(since Windows 7)

[PascalDragon]

[…] to me, it would have been (since 2012-14) quiet simple to add a function for example named 'HardRand(limit: longint)' or a compiler directive like '$RNDGEN ON|OFF' modifying the behavior of std 'Random' call

Well, if it so simple why don’t you file a merge request and the developers will incorporate it or turn it down for reasons they explain.

I'd probably simply copy your reasons

[Thaddy]

It would also be futile for the reasons I mentioned:
https://en.wikipedia.org/wiki/RDRAND
https://www.intel.com/content/dam/develop/external/us/en/documents/drng-software-implementation-guide-2-1-185467.pdf
and
https://www.microsoft.com/en-us/security/blog/2019/11/25/going-in-depth-on-the-windows-10-random-number-generation-infrastructure/

On both Windows and Unix'es if the hardware random instructions are available for the CPU they will be used for the entropy pool for a couple of years now. And I showed how to test it.

A summary from Microsoft:

Code: Text  [Select][+][-]

1. RDRAND/RDSEED
2. The Intel RDRAND instruction is an on-demand high quality source of random data.
3. If the RDRAND instruction is present, Winload gathers 256 bits of entropy from the
4. RDRAND instruction. Similarly, our kernel-mode code creates a high-pull source that
5. provides 512 bits of entropy from the RDRAND instruction for each reseed. (As a high
6. source, the first 256 bits are always put in pool 0; providing 512 bits ensures that the
7. other pools also get entropy from this source.)
8. Due to some unfortunate design decisions in the internal RDRAND logic, the RDRAND
9. instruction only provides random numbers with a 128-bit security level. The Win10 code
10. tries to work around this limitation by gathering a large amount of output from RDRAND
11. which should trigger a reseed of the RDRAND-internal PRNG to get more entropy.
12. Whilst this solves the problem in most cases, it is possible for another thread to gather
13. similar outputs form RDRAND which means that a 256-bit security level cannot be
14. guaranteed.
15. Based on our feedback about this problem, Intel implemented the RDSEED instruction
16. that gives direct access to the internal entropy source. When the RDSEED instruction is
17. present, it is used in preference to RDRAND instruction which avoids the problem and
18. provides the full desired guarantees. For each reseed, we gather 128 output bytes from
19. RDSEED, hash them with SHA-512 to produce 64 output bytes. As explained before, 32
20. of these go into pool 0 and the others into the ‘next’ pool for this entropy source.

Basically - apart from embedded systems - it is bollocks to patch a programming language for this.
An additional check for linux which should return either 128 (old),256 or 512 (rare):

Code: Bash  [Select][+][-]

1. cat /proc/sys/kernel/random/entropy_avail