Be careful with shr

[tetrastes]

Why should I read the warnings of the C compilers?

To know something new.

They conform to C standard:

Who told you that?

For example http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf, page 84.

So why do you think that in Pascal this result must be defined?

Because the problem is not in the result of the assembly shift instruction.

And what? For C the problem IS in the result of the assembly shift instruction?
The standard of high-level language is not defined by assembly instructions of specific architecture.

If you test the code I posted in my previous post. Pass 64 and you get 0 (as expected).

32-bit program gives 0, 64-bit 18446744073709551615. The result is undefined. 

[engkin]

When you pass constants like in:

Code: Pascal  [Select][+][-]

1.   l := high(l);
2.   l := l shr ((sizeof(l) * 8));

FPC "simplifies" the expression and does constant folding:

Code: Pascal  [Select][+][-]

1. //unit compiler\nmat.pas
2.     function tshlshrnode.simplify(forinline : boolean):tnode;
3.       var
4.         lvalue,rvalue : Tconstexprint;
5.       begin
6.         result:=nil;
7.         { constant folding }
8.         if is_constintnode(right) then
9.           begin
10.             if forinline then
11.               begin
12.                 { shl/shr are unsigned operations, so cut off upper bits }
13.                 case resultdef.size of
14. ....
15.                   8:
16.                     rvalue:=tordconstnode(right).value and byte($3f);
17.                   else
18.  

in the process it assumes the number of bits to shift should be reduced when you shift an 8 byte number to only first 6 bits. This turns the code into:

Code: Pascal  [Select][+][-]

1.   l := high(l) shr (64 and $3F);

64 and $3F equals 0 and results in high(l)

Edit:
Corrected var name.

[engkin]

If you test the code I posted in my previous post. Pass 64 and you get 0 (as expected).

32-bit program gives 0, 64-bit 18446744073709551615. The result is undefined. 

I don't have a problem if the result is undefined in 64-bit as long as it comes from the assembly. I can not find in my copy of "IA-32 Intel Architecture Developer’s Manual - Volume 2 - Instruction Set Reference" that SHR is not defined for values over 31. Since I am on a 32-bit system I can not tell, but it could be another bug for FPC 64-bit code.

Edit:

Why should I read the warnings of the C compilers?

To know something new.

They conform to C standard:

Who told you that?

For example http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf, page 84.

You misunderstood my point. This is Pascal forum. FPC does not conform to C standard. There is not reason from me to learn something new about C here.

So why do you think that in Pascal this result must be defined?

Because the problem is not in the result of the assembly shift instruction.

And what? For C the problem IS in the result of the assembly shift instruction?
The standard of high-level language is not defined by assembly instructions of specific architecture.

You might be right about C, you should discuss it in a C related forum. FYI, my C 32-bit compiler gave the expected results on unsigned long long.

[Thaddy]

When you have a modern C compiler and FPC 3+ The bit patterns are the same.
And as stated: don't mix up unsigned(31/63) and signed(32/64). C gives you a warning. FreePascal circumvents this warning and shifts in zero's. Which is correct. OTOH: shl will give a neat overflow when shift is larger than size.{$Q+}

[engkin]

When you have a modern C compiler and FPC 3+ The bit patterns are the same.

What do you mean?

And as stated: don't mix up unsigned(31/63) and signed(32/64). C gives you a warning. FreePascal circumvents this warning and shifts in zero's. Which is correct.

What mix do you see in this code:

Code: Pascal  [Select][+][-]

1. var
2.   l: uint64;
3. begin
4.   l := high(l);
5.   l := l shr ((sizeof(l) * 8));
6.   writeln(l);

[ASerge]

As I understand, the problem is that High(UInt64) shr 64 produces zero in the 32-bit compiler.
The Delphi documentation and the Intel documentation say that nothing is done in this case.

Code: Pascal  [Select][+][-]

1. program Project1;
2. {$APPTYPE CONSOLE}
3. {$IFDEF FPC}{$MODE OBJFPC}{$ENDIF}
4.  
5. var
6.   L: UInt64 = High(L);
7.   N: Integer = 64;
8. begin
9.   L := L shr N;
10.   Writeln(L);
11.   Readln;
12. end.

Result on Windows:
Delphi 7 x32: High(UInt64) (use -1, because compiler support only Int64 constants)
Delphi 10.2 x32: High(UInt64)
FPC 3.0.4 x32: Zero
FPC 3.0.4 x64: High(UInt64)

[jamie]

The last time I did ASSEMBLER with shifts it is a known fact that intel processors
do not use the full operand for the shift..

 In 32 bit mode only the lower 5 bits are used.

 This of course will equal 32 if you do this, 2^5= 32 but, that also includes the
 value 0 which does not do any shifting of course.
  So what you end up with is a 31 shift level.

  so what ever the operand value is   would be this
   Shift_Amount := Operand and $1F;

  so if you do have overflows, expect the lower 5 bits only.

 In the case of the BYTE and most likely even a WORD type, I would expect it to result to
 0 with a 32 bit target, only because the bit limit can cover this.

 with 64 processor I assume the bit limit is raised to 6 bits which still gives you the issue of
only being able to shift using the lower 6 bits = 63 times.

[engkin]

The last time I did ASSEMBLER with shifts it is a known fact that intel processors
do not use the full operand for the shift..

 In 32 bit mode only the lower 5 bits are used.

While on ARM processors (LSR logical shift right):

If n is 32 or more, then all the bits in the result are cleared to 0.

[ASerge]

The last time I did ASSEMBLER with shifts it is a known fact that intel processors
do not use the full operand for the shift..

As I pointed out, this is clearly described in the Intel documentation for SHR:

The destination operand can be a register or a memory location. The count operand can be an immediate value or the CL register. The count is masked to 5 bits (or 6 bits if in 64-bit mode and REX.W is used). The count range is limited to 0 to 31 (or 63 if 64-bit mode and REX.W is used).

And in delphi:

The operations x shl y and x shr y shift the value of x to the left or right by y bits, which (if x is an unsigned integer) is equivalent to multiplying or dividing x by 2^y; the result is of the same type as x. For example, if N stores the value 01101 (decimal 13), then N shl 1 returns 11010 (decimal 26). Note that the value of y is interpreted modulo the size of the type of x. Thus for example, if x is an integer, x shl 40 is interpreted as x shl 8 because an integer is 32 bits and 40 mod 32 is 8.

But... The use of 64-bit integers in 32-bit mode assumes a SHRD operation for which the Intel documentation reports:

In non-64-bit modes and default 64-bit mode, the width of the count mask is 5 bits. Only bits 0 through 4 of the count register are used (masking the count to a value between 0 and 31). If the count is greater than the operand size, the result is undefined.

FPC documentation says only:

shr - Bitwise shift to the right

[BrunoK]

And in delphi:

The operations x shl y and x shr y shift the value of x to the left or right by y bits, which (if x is an unsigned integer) is equivalent to multiplying or dividing x by 2^y; the result is of the same type as x. For example, if N stores the value 01101 (decimal 13), then N shl 1 returns 11010 (decimal 26). Note that the value of y is interpreted modulo the size of the type of x. Thus for example, if x is an integer, x shl 40 is interpreted as x shl 8 because an integer is 32 bits and 40 mod 32 is 8.

FPC 3.0.4 behaves quite like it is described in the Delphi documentation. But depending on the optimization level and the way of doing the shift, result will not be the same.

Code: Pascal  [Select][+][-]

1. program prjShr31_WriteLn_1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   Classes { you can add units after this };
7.  
8. var
9.   ui64: uint64;
10.  
11. procedure OPTI_64_Var;
12. var
13.   ShiftVar : integer;
14. begin
15.   ShiftVar := SizeOf(ui64) * 8; // + 2; to show difference depending on Optimization level
16.   ui64 := ui64 shr ShiftVar;
17.   writeln('OPTI_64_Var -> ui64 shr ui64 shr ShiftCount :', ui64);
18.   // Results for Optimizations :  FPC 3.0.4 for Intel 32 bit on 64 bit windows 10
19.   //   None    : 0
20.   //   Level 1 : 0
21.   //   Level 2 : 0
22.   //   Level 3 : 18446744073709551615
23.   //   Level 4 : 18446744073709551615
24. end;
25.  
26. procedure OPTI_64_Const;
27. Const
28.   ShiftCONST = SizeOf(ui64) * 8; // + 2; to show difference depending on Optimization level
29.   // ^- This is a CONST !
30. begin
31.   ui64 := ui64 shr ShiftCONST;
32.   writeln('OPTI_64_Const -> ui64 shr 64 :', ui64);
33.   // Results for Optimizations :  FPC 3.0.4 for Intel 32 bit on 64 bit windows 10
34.   //   None    : 18446744073709551615
35.   //   Level 1 : 18446744073709551615
36.   //   Level 2 : 18446744073709551615
37.   //   Level 3 : 18446744073709551615
38.   //   Level 4 : 18446744073709551615
39. end;
40.  

Up to optimization level o2, the "ui64 shr ShiftVar" will always result in 0 if  ShiftCount>=64 whereas   "ui64 shr ShiftCONST" (for ShiftCONST>=64 will be equivalent to  "ui64 shr (ShiftCONST mod 64)"

Identical logic, inconsistent results.

[tetrastes]

You might be right about C, you should discuss it in a C related forum. FYI, my C 32-bit compiler gave the expected results on unsigned long long.

And what results it gave for unsigned long? Expected or not?
Also change code in your example from "Uint64 shr 64" to "DWord shr 32". And "DWord shr 33", "DWord shr 34"...
What results do you expect? Zero, I suppose? 

[Blaazen]

In non-64-bit modes and default 64-bit mode, the width of the count mask is 5 bits. Only bits 0 through 4 of the count register are used (masking the count to a value between 0 and 31). If the count is greater than the operand size, the result is undefined.

So internally should compiler do

Code: Pascal  [Select][+][-]

1. l := l shr 31;
2. l := l shr 31;
3. l := l shr 2;
4.  

to do 64-bit shift safely?

[ASerge]

...to do 64-bit shift safely?

For compatibility with the x64 modes and Delphi write like this:

Code: Pascal  [Select][+][-]

1. L := L shr (N mod SizeOf(L));

[tetrastes]

...to do 64-bit shift safely?

For compatibility with the x64 modes and Delphi write like this:

Code: Pascal  [Select][+][-]

1. L := L shr (N mod SizeOf(L));

Code: Pascal  [Select][+][-]

1. L := L shr (N mod (SizeOf(L) * 8));

as I understand.
But for what?

[ASerge]

Code: Pascal  [Select][+][-]

1. L := L shr (N mod (SizeOf(L) * 8));

as I understand.
But for what?

Thanks for the correction.
As mentioned above "For compatibility with the x64 modes and Delphi write like this". FPC x32 generate not compatible code.