### Bookstore

 Computer Math and Games in Pascal (preview) Lazarus Handbook

### Author Topic: Result mismatch on division by exponential notation and literal notation.  (Read 1206 times)

#### smaniok

• Newbie
• Posts: 3
##### Result mismatch on division by exponential notation and literal notation.
« on: August 09, 2020, 05:30:21 am »
Hello everyone.

I found an inconsistency in results when dividing a float number by an integer in exponential notation and literal notation.

Ex, the following code used to round and crop a float number to six decimal places, using the division first by 1E6 (exponential notation) and last for 1000000 (literal notation as I mean):

Code: Pascal  [Select][+][-]
1. program NumberTest;
2.
3. const
4.   NUM_CONST = 2.123456789;
5. var
6.   NumVar: Double;
7.   Res1, Res2, Res3, Res4: Double;
8. begin
9.   NumVar := NUM_CONST;
10.   Res1 := Round(NUM_CONST * 1E6) / 1E6;
11.   Res2 := Round(NumVar    * 1E6) / 1E6;
12.   Res3 := Round(NUM_CONST * 1E6) / 1000000;
13.   Res4 := Round(NumVar    * 1E6) / 1000000;
14.
15.   WriteLn('Res1 = ', Res1);
16.   WriteLn('Res2 = ', Res2);
17.   WriteLn('Res3 = ', Res3);
18.   WriteLn('Res4 = ', Res4);
19. end.
20.

I expected the same result in both, but it is not the case.

The result output on Windows 10 (64 bit) is:
Quote
Res1 =  2.1234570000000002E+000
Res2 =  2.1234569549560547E+000
Res3 =  2.1234570000000002E+000
Res4 =  2.1234570000000002E+000

Note that the operation using the const number results as expected (2.123457),  but when using a variable it results in (2.1234569549560547).

The same code on Linux (Ubuntu 20.04) results in:
Quote
Res1 =  2.1234569549560547E+000
Res2 =  2.1234569549560547E+000
Res3 =  2.1234570000000001E+000
Res4 =  2.1234570000000001E+000

Note that on Linux, the operations using const and var result the same (2.1234569549560547).

Why does an operation using the expenential notation differ from using the literal one?

Edit: I am using Free Pascal 3.2.0, but the same occurs in 3.0.4
« Last Edit: August 09, 2020, 05:43:39 am by smaniok »

#### julkas

• Guest
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #1 on: August 09, 2020, 11:13:27 am »
Try also following code -
Code: Pascal  [Select][+][-]
1. program NumberTest;
2. {\$IFDEF FPC}
3. {\$MODE Delphi}
4. {\$ENDIF}
5. uses SysUtils;
6.
7.     const
8.       NUM_CONST = 2.123456789;
9.     var
10.       NumVar: Double;
11.       Res1, Res2, Res3, Res4: Double;
12.     begin
13.       NumVar := NUM_CONST;
14.       Res1 := Round(NUM_CONST * 1E6);
15.       Res2 := Round(NumVar    * 1E6);
16.       WriteLn(PUInt64(@Res1)^);
17.       WriteLn(PUInt64(@Res2)^);
18.
19.       Res1 := Res1 / 1E6;
20.       Res2 := Res2 / 1E6;
21.       WriteLn(PUInt64(@Res1)^);
22.       WriteLn(PUInt64(@Res2)^);
23.
24.       Res3 := Round(NUM_CONST * 1E6) / 1000000;
25.       Res4 := Round(NumVar    * 1E6) / 1000000;
26.
27.       WriteLn('Res1 = ', Res1);
28.       WriteLn('Res2 = ', Res2);
29.       WriteLn('Res3 = ', Res3);
30.       WriteLn('Res4 = ', Res4);
31.
32.       Res1 := Round(NUM_CONST * 1E6) / 1E6;
33.       Res2 := Round(NumVar    * 1E6) / 1E6;
34.       WriteLn(PUInt64(@Res1)^);
35.       WriteLn(PUInt64(@Res2)^);
36.       WriteLn('Res1 = ', Res1);
37.       WriteLn('Res2 = ', Res2);
38.
39.     end.
40.
FPC output -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4701814500532158464
3. 4701814500532158464
4. 4611964018876986044
5. 4611964018876986044
6. Res1 =  2.1234570000000002E+000
7. Res2 =  2.1234570000000002E+000
8. Res3 =  2.1234570000000002E+000
9. Res4 =  2.1234570000000002E+000
10. 4611964018876986044
11. 4611964018775556096
12. Res1 =  2.1234570000000002E+000
13. Res2 =  2.1234569549560547E+000
14.
Delphi output -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4701814500532158464
3. 4701814500532158464
4. 4611964018876986044
5. 4611964018876986044
6. Res1 =  2.12345700000000E+0000
7. Res2 =  2.12345700000000E+0000
8. Res3 =  2.12345700000000E+0000
9. Res4 =  2.12345700000000E+0000
10. 4611964018876986044
11. 4611964018876986044
12. Res1 =  2.12345700000000E+0000
13. Res2 =  2.12345700000000E+0000
14.

#### ASerge

• Hero Member
• Posts: 1671
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #2 on: August 09, 2020, 11:21:38 am »
I found an inconsistency in results when dividing a float number by an integer in exponential notation and literal notation.
For the 1E6 constant, the compiler uses the Single value type. And calculations of the corresponding accuracy. When an integer is used (1000000), the constant is converted to the Double type, and double-precision calculations are applied accordingly.

#### julkas

• Guest
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #3 on: August 09, 2020, 11:57:06 am »
@ASerge How you explain this -
Code: Pascal  [Select][+][-]
1. program NumberTest;
2. {\$IFDEF FPC}
3. {\$MODE Delphi}
4. {\$ENDIF}
5.
6.     var
7.       NumVar: Double;
8.       Res1: Double;
9.     begin
10.       NumVar := 2.123456789;
11.
12.       Res1 := Round(NumVar * 1.0) / 1E6;
13.       WriteLn(PUInt64(@Res1)^);
14.
15.           Res1 := Round(NumVar * 1.0);
16.           Res1 := Res1 / 1E6;
17.       WriteLn(PUInt64(@Res1)^);
18.     end.
19.

FPC result -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4521832792723554304
3. 4521832792735477133
4.

Delphi -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4521832792735477133
3. 4521832792735477133
4.

FPC bug ?
« Last Edit: August 09, 2020, 12:08:35 pm by julkas »

#### julkas

• Guest
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #4 on: August 09, 2020, 02:11:14 pm »
Another -
Code: Pascal  [Select][+][-]
1. program NumberTest;
2. {\$IFDEF FPC}
3. {\$MODE Delphi}
4. {\$ENDIF}
5. uses Math;
6.
7.     var
8.       NumVar: Double;
9.
10.     begin
11.       NumVar := 2.0;
12.
13.       NumVar := Round(NumVar) / 10.0;
14.       WriteLn(PUInt64(@NumVar)^);
15.
16.       NumVar := Round(Double(2.0)) / 10.0;
17.       WriteLn(PUInt64(@NumVar)^);
18.
19.     end.
20.
Result -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4596373779801702400
3. 4596373779694328218
4.
C lang - https://ideone.com/SKRJ2Q
« Last Edit: August 09, 2020, 02:40:42 pm by julkas »

#### ASerge

• Hero Member
• Posts: 1671
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #5 on: August 09, 2020, 08:54:18 pm »
@ASerge How you explain this -
Same. Change 1E6 to Double(1E6).

#### smaniok

• Newbie
• Posts: 3
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #6 on: August 09, 2020, 11:16:32 pm »
For the 1E6 constant, the compiler uses the Single value type. And calculations of the corresponding accuracy. When an integer is used (1000000), the constant is converted to the Double type, and double-precision calculations are applied accordingly.

Yes, typecast solved the difference. In addition, converting the left operand also has the same effect.

Code: Bash  [Select][+][-]
1. Res2 := Double(Round(NumVar * 1E6)) / 1E6;
2.

@ASerge How you explain this -
Code: Pascal  [Select][+][-]
1. program NumberTest;
2. {\$IFDEF FPC}
3. {\$MODE Delphi}
4. {\$ENDIF}
5.
6.     var
7.       NumVar: Double;
8.       Res1: Double;
9.     begin
10.       NumVar := 2.123456789;
11.
12.       Res1 := Round(NumVar * 1.0) / 1E6;
13.       WriteLn(PUInt64(@Res1)^);
14.
15.           Res1 := Round(NumVar * 1.0);
16.           Res1 := Res1 / 1E6;
17.       WriteLn(PUInt64(@Res1)^);
18.     end.
19.

FPC result -
Code: Bash  [Select][+][-]
1. \$ ./consta.exe
2. 4521832792723554304
3. 4521832792735477133
4.

The same here, Res1 is explicitly declared as Double.

#### jamie

• Hero Member
• Posts: 3538
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #7 on: August 10, 2020, 12:58:53 am »
Just move on, don't expect this to ever get fixed.

Get use to Type casting the hell out of everything or do your coding in Delphi and save the agony of bug hunting.

The only true wisdom is knowing you know nothing

#### ASerge

• Hero Member
• Posts: 1671
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #8 on: August 10, 2020, 06:15:56 pm »
The same here, Res1 is explicitly declared as Double.
Constant (literal)! Not a variable, result, or anything else. The constant is defined as the Single type.

• Hero Member
• Posts: 10482
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #9 on: August 10, 2020, 07:26:23 pm »
Constant (literal)! Not a variable, result, or anything else. The constant is defined as the Single type.
In that case it is a bug. An untyped const should take the highest possible resolution for the platform.
Plz report.
When you ask a question that is actually answered in the documentation, you are either lazy or a moron.

#### ASerge

• Hero Member
• Posts: 1671
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #10 on: August 10, 2020, 07:53:35 pm »
In that case it is a bug. An untyped const should take the highest possible resolution for the platform.
Is this documented somewhere?
I have seen other things that the Single type is the only one that is guaranteed on all platforms.

#### Jonas Maebe

• Hero Member
• Posts: 757
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #11 on: August 10, 2020, 08:41:59 pm »
See https://wiki.freepascal.org/User_Changes_2.2.0#Floating_point_constants for more information. Back then, the current behaviour was considered Delphi-compatible (but perhaps an error was made). It also mentions a way to specify a higher minimal precision for floating point constants.

• Hero Member
• Posts: 10482
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #12 on: August 10, 2020, 09:06:37 pm »
See https://wiki.freepascal.org/User_Changes_2.2.0#Floating_point_constants for more information. Back then, the current behaviour was considered Delphi-compatible (but perhaps an error was made). It also mentions a way to specify a higher minimal precision for floating point constants.
So that should be reverted.
When you ask a question that is actually answered in the documentation, you are either lazy or a moron.

#### Jonas Maebe

• Hero Member
• Posts: 757
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #13 on: August 10, 2020, 09:23:42 pm »
More likely refined.

#### Alextp

• Hero Member
• Posts: 1121
##### Re: Result mismatch on division by exponential notation and literal notation.
« Reply #14 on: August 11, 2020, 06:53:56 pm »
>An untyped const should take the highest possible resolution for the platform.
Plz report.

Do I need to write to bugtracker about it? or Jonas made a fix?