Recent

Author Topic: Result mismatch on division by exponential notation and literal notation.  (Read 3554 times)

smaniok

  • Newbie
  • Posts: 3
Hello everyone.

I found an inconsistency in results when dividing a float number by an integer in exponential notation and literal notation.

Ex, the following code used to round and crop a float number to six decimal places, using the division first by 1E6 (exponential notation) and last for 1000000 (literal notation as I mean):

Code: Pascal  [Select][+][-]
  1. program NumberTest;
  2.  
  3. const
  4.   NUM_CONST = 2.123456789;
  5. var
  6.   NumVar: Double;
  7.   Res1, Res2, Res3, Res4: Double;
  8. begin
  9.   NumVar := NUM_CONST;
  10.   Res1 := Round(NUM_CONST * 1E6) / 1E6;
  11.   Res2 := Round(NumVar    * 1E6) / 1E6;
  12.   Res3 := Round(NUM_CONST * 1E6) / 1000000;
  13.   Res4 := Round(NumVar    * 1E6) / 1000000;
  14.  
  15.   WriteLn('Res1 = ', Res1);
  16.   WriteLn('Res2 = ', Res2);
  17.   WriteLn('Res3 = ', Res3);
  18.   WriteLn('Res4 = ', Res4);
  19. end.
  20.  

I expected the same result in both, but it is not the case.

The result output on Windows 10 (64 bit) is:
Quote
Res1 =  2.1234570000000002E+000
Res2 =  2.1234569549560547E+000
Res3 =  2.1234570000000002E+000
Res4 =  2.1234570000000002E+000

Note that the operation using the const number results as expected (2.123457),  but when using a variable it results in (2.1234569549560547).

The same code on Linux (Ubuntu 20.04) results in:
Quote
Res1 =  2.1234569549560547E+000
Res2 =  2.1234569549560547E+000
Res3 =  2.1234570000000001E+000
Res4 =  2.1234570000000001E+000

Note that on Linux, the operations using const and var result the same (2.1234569549560547).

Why does an operation using the expenential notation differ from using the literal one?

Edit: I am using Free Pascal 3.2.0, but the same occurs in 3.0.4
« Last Edit: August 09, 2020, 05:43:39 am by smaniok »

julkas

  • Guest
Try also following code -
Code: Pascal  [Select][+][-]
  1. program NumberTest;
  2. {$IFDEF FPC}
  3. {$MODE Delphi}
  4. {$ENDIF}
  5. uses SysUtils;
  6.  
  7.     const
  8.       NUM_CONST = 2.123456789;
  9.     var
  10.       NumVar: Double;
  11.       Res1, Res2, Res3, Res4: Double;
  12.     begin
  13.       NumVar := NUM_CONST;
  14.       Res1 := Round(NUM_CONST * 1E6);
  15.       Res2 := Round(NumVar    * 1E6);
  16.       WriteLn(PUInt64(@Res1)^);
  17.       WriteLn(PUInt64(@Res2)^);  
  18.  
  19.       Res1 := Res1 / 1E6;
  20.       Res2 := Res2 / 1E6;
  21.       WriteLn(PUInt64(@Res1)^);
  22.       WriteLn(PUInt64(@Res2)^);  
  23.  
  24.       Res3 := Round(NUM_CONST * 1E6) / 1000000;
  25.       Res4 := Round(NumVar    * 1E6) / 1000000;
  26.  
  27.       WriteLn('Res1 = ', Res1);
  28.       WriteLn('Res2 = ', Res2);
  29.       WriteLn('Res3 = ', Res3);
  30.       WriteLn('Res4 = ', Res4);
  31.  
  32.       Res1 := Round(NUM_CONST * 1E6) / 1E6;
  33.       Res2 := Round(NumVar    * 1E6) / 1E6;
  34.       WriteLn(PUInt64(@Res1)^);
  35.       WriteLn(PUInt64(@Res2)^);  
  36.       WriteLn('Res1 = ', Res1);
  37.       WriteLn('Res2 = ', Res2);
  38.                  
  39.     end.
  40.  
FPC output -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4701814500532158464
  3. 4701814500532158464
  4. 4611964018876986044
  5. 4611964018876986044
  6. Res1 =  2.1234570000000002E+000
  7. Res2 =  2.1234570000000002E+000
  8. Res3 =  2.1234570000000002E+000
  9. Res4 =  2.1234570000000002E+000
  10. 4611964018876986044
  11. 4611964018775556096
  12. Res1 =  2.1234570000000002E+000
  13. Res2 =  2.1234569549560547E+000
  14.  
Delphi output -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4701814500532158464
  3. 4701814500532158464
  4. 4611964018876986044
  5. 4611964018876986044
  6. Res1 =  2.12345700000000E+0000
  7. Res2 =  2.12345700000000E+0000
  8. Res3 =  2.12345700000000E+0000
  9. Res4 =  2.12345700000000E+0000
  10. 4611964018876986044
  11. 4611964018876986044
  12. Res1 =  2.12345700000000E+0000
  13. Res2 =  2.12345700000000E+0000
  14.  

ASerge

  • Hero Member
  • *****
  • Posts: 2214
I found an inconsistency in results when dividing a float number by an integer in exponential notation and literal notation.
For the 1E6 constant, the compiler uses the Single value type. And calculations of the corresponding accuracy. When an integer is used (1000000), the constant is converted to the Double type, and double-precision calculations are applied accordingly.

julkas

  • Guest
@ASerge How you explain this -
Code: Pascal  [Select][+][-]
  1. program NumberTest;
  2. {$IFDEF FPC}
  3. {$MODE Delphi}
  4. {$ENDIF}
  5.  
  6.     var
  7.       NumVar: Double;
  8.       Res1: Double;
  9.     begin
  10.       NumVar := 2.123456789;
  11.          
  12.       Res1 := Round(NumVar * 1.0) / 1E6;
  13.       WriteLn(PUInt64(@Res1)^);
  14.          
  15.           Res1 := Round(NumVar * 1.0);
  16.           Res1 := Res1 / 1E6;
  17.       WriteLn(PUInt64(@Res1)^);
  18.     end.
  19.  

FPC result -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4521832792723554304
  3. 4521832792735477133
  4.  

Delphi -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4521832792735477133
  3. 4521832792735477133
  4.  

FPC bug ?
« Last Edit: August 09, 2020, 12:08:35 pm by julkas »

julkas

  • Guest
Another - 
Code: Pascal  [Select][+][-]
  1. program NumberTest;
  2. {$IFDEF FPC}
  3. {$MODE Delphi}
  4. {$ENDIF}
  5. uses Math;
  6.  
  7.     var
  8.       NumVar: Double;
  9.  
  10.     begin
  11.       NumVar := 2.0;
  12.                  
  13.       NumVar := Round(NumVar) / 10.0;
  14.       WriteLn(PUInt64(@NumVar)^);
  15.  
  16.       NumVar := Round(Double(2.0)) / 10.0;
  17.       WriteLn(PUInt64(@NumVar)^);
  18.  
  19.     end.
  20.  
Result -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4596373779801702400
  3. 4596373779694328218
  4.  
C lang - https://ideone.com/SKRJ2Q
« Last Edit: August 09, 2020, 02:40:42 pm by julkas »

ASerge

  • Hero Member
  • *****
  • Posts: 2214
@ASerge How you explain this -
Same. Change 1E6 to Double(1E6).

smaniok

  • Newbie
  • Posts: 3
For the 1E6 constant, the compiler uses the Single value type. And calculations of the corresponding accuracy. When an integer is used (1000000), the constant is converted to the Double type, and double-precision calculations are applied accordingly.

Yes, typecast solved the difference. In addition, converting the left operand also has the same effect.

Code: Bash  [Select][+][-]
  1. Res2 := Double(Round(NumVar * 1E6)) / 1E6;
  2.  

@ASerge How you explain this -
Code: Pascal  [Select][+][-]
  1. program NumberTest;
  2. {$IFDEF FPC}
  3. {$MODE Delphi}
  4. {$ENDIF}
  5.  
  6.     var
  7.       NumVar: Double;
  8.       Res1: Double;
  9.     begin
  10.       NumVar := 2.123456789;
  11.          
  12.       Res1 := Round(NumVar * 1.0) / 1E6;
  13.       WriteLn(PUInt64(@Res1)^);
  14.          
  15.           Res1 := Round(NumVar * 1.0);
  16.           Res1 := Res1 / 1E6;
  17.       WriteLn(PUInt64(@Res1)^);
  18.     end.
  19.  

FPC result -
Code: Bash  [Select][+][-]
  1. $ ./consta.exe
  2. 4521832792723554304
  3. 4521832792735477133
  4.  


The same here, Res1 is explicitly declared as Double.

jamie

  • Hero Member
  • *****
  • Posts: 6077
Just move on, don't expect this to ever get fixed.

Get use to Type casting the hell out of everything or do your coding in Delphi and save the agony of bug hunting.

The only true wisdom is knowing you know nothing

ASerge

  • Hero Member
  • *****
  • Posts: 2214
The same here, Res1 is explicitly declared as Double.
Constant (literal)! Not a variable, result, or anything else. The constant is defined as the Single type.

Thaddy

  • Hero Member
  • *****
  • Posts: 14172
  • Probably until I exterminate Putin.
Constant (literal)! Not a variable, result, or anything else. The constant is defined as the Single type.
In that case it is a bug. An untyped const should take the highest possible resolution for the platform.
Plz report.
Specialize a type, not a var.

ASerge

  • Hero Member
  • *****
  • Posts: 2214
In that case it is a bug. An untyped const should take the highest possible resolution for the platform.
Is this documented somewhere?
I have seen other things that the Single type is the only one that is guaranteed on all platforms.

Jonas Maebe

  • Hero Member
  • *****
  • Posts: 1058
See https://wiki.freepascal.org/User_Changes_2.2.0#Floating_point_constants for more information. Back then, the current behaviour was considered Delphi-compatible (but perhaps an error was made). It also mentions a way to specify a higher minimal precision for floating point constants.

Thaddy

  • Hero Member
  • *****
  • Posts: 14172
  • Probably until I exterminate Putin.
See https://wiki.freepascal.org/User_Changes_2.2.0#Floating_point_constants for more information. Back then, the current behaviour was considered Delphi-compatible (but perhaps an error was made). It also mentions a way to specify a higher minimal precision for floating point constants.
So that should be reverted.
Specialize a type, not a var.

Jonas Maebe

  • Hero Member
  • *****
  • Posts: 1058
More likely refined.

AlexTP

  • Hero Member
  • *****
  • Posts: 2365
    • UVviewsoft
>An untyped const should take the highest possible resolution for the platform.
Plz report.

Do I need to write to bugtracker about it? or Jonas made a fix?


 

TinyPortal © 2005-2018