How to hide IEEE754 limitations

[Tommi]

Hello everyone,
I am actually writing a calculator software.
If I use any calculator software and I write : 32.47875-32, I obtain 0.47875.
If I do it with mine (that uses "double" type) I obtain : 0.478749999999998 .

So the question is: how normal calculators works ?

May be using strings instead floating point ? I mean stacking numbers and operating like an human do with paper and pen ?

[Handoko]

Maybe you need Benibela's Big Decimal Math:
https://github.com/benibela/bigdecimalmath?tab=readme-ov-file

or Bart's Big Numbers:
https://forum.lazarus.freepascal.org/index.php?topic=22016.0

[Lulu]

or have you tried with Extended type ?
https://www.freepascal.org/docs-html/3.0.0/ref/refsu6.html

[Thaddy]

Extended is not really a recommended type for such things, since it is platform.
What you can do instead is use the currency type for a simple calculator, because that is already a scaled integer type.
Rounding will look more natural to the eye.
For higher precision look at the big integer types like Handoko mentioned.

[wp]

The problem is that the data type "double" (and "single" and "extended" as well) has only a given number of bits to represent a number. Only numbers which can be expressed as a combination of powers of 2 are exact, for anything else the closest combination of powers of 2 is used. Expressed with extreme words. The exact number 0.47875 simply does not "exist" in the computer.

In most calculations, ultimate accuracy as offered by big-number libraries is not needed (and even they cannot express the result of 1/3 in an "exact" way because this would require an infinite number of decimal places). Your issue is not the precision of the numbers (which will almost never be "exact"), but the way how these numbers are displayed. And for this you only need to use the correct formatting functions.

You should specify in your calculator application the maximum number of decimal places and then format every numeric output to it. When, for example, there is a maximum of 6 decimal places, you should format the calculation result x by FormatFloat('0.######', x) or by Format('%.6g', [ x ]):

Code: Pascal  [Select][+][-]

1. program Project1;
2. uses
3.   SysUtils;
4. var
5.   x: Double;
6. begin
7.   x := 32.47875-32;
8.   WriteLn(x);
9.   WriteLn(FormatFloat('0.######', x));
10.   WriteLn(Format('%.6g', [x]));
11.   WriteLn;
12.   x := 1.0/3;
13.   WriteLn(x);
14.   WriteLn(FormatFloat('0.######', x));
15.   WriteLn(Format('%.6g', [x]));
16.   WriteLn;
17.   x := 2.5+0.1;
18.   WriteLn(x);
19.   WriteLn(FormatFloat('0.######', x));
20.   WriteLn(Format('%.6g', [x]));
21.   WriteLn;
22.   ReadLn;
23. end.
24.  

Code: [Select]

Output:
 4.7875000000000001E-001
0.47875
0.47875

 3.3333334326744080E-001
0.333333
0.333333

 2.6000000000000001E+000
2.6
2.6You could also use the format string '0.000000' for FormatFloat and '%.6f' for Format to get also nice rounding, but this would add unnecessary zeros as decimals.

[Hartmut]

If your calculator always needs to display only numbers in a "medium" range, than a fixed number of decimals might be a satisfying solution. But if it should be able to display very big results, which have e.g. 15 digits (or more) left of the decimal dot or to display very small results like 0.000000000012345, then I think a fixed number of decimals would not be a good solution.

What I did for such cases: I created a formatting function, where I can limit the total number of displayed digits and can limit the maximum displayed digits right of the decimal dot:

Code: Pascal  [Select][+][-]

1. function int_range(x,min,max: integer): integer;
2.    {forces 'x' into the range of [min..max]}
3.    begin
4.    if x < min then exit(min) else
5.       if x > max then exit(max) else exit(x);
6.    end;
7.  
8. function digits_left(d: double): integer;
9.    {returns the number of digits of 'd' which are left to the decimal dot}
10.    var s: string;
11.    begin
12.    str(abs(d):0:38, s);
13.    exit(pos('.',s));
14.    end;
15.  
16. function double_to_total_digits(d: double; digits,decimals: integer): string;
17.    {formats a double to a string.
18.     I: digits: max. total number of digits (left and right of the decimal dot)
19.        decimals: max. number of digits right of the decimal dot}
20.    var s: string;
21.    begin
22.    str(d:0:int_range(digits-digits_left(d), 0,decimals), s);
23.    exit(s);
24.    end;

The benefit is: if you want, you can show big results with less decimals and small results with more decimals. For 'double' I would recommend to use a total of 16 digits.

Code: Pascal  [Select][+][-]

1. begin
2. writeln(double_to_total_digits(1234567890123 + 2/3, 16,16));
3. writeln(double_to_total_digits((1234567890123 + 2/3) * 1E-20, 16,16));
4. end.

Code: Text  [Select][+][-]

1. 1234567890123.667  // gives only 3 digits right of the decimal dot
2. 0.000000012345679  // gives 15 digits right of the decimal dot

[Thaddy]

Much, much simpler:

Code: Pascal  [Select][+][-]

1. begin
2.   writeln(32.47875-32:1:5);// write/writeln has format specifiers....
3. end.

Complete program, believe it or not.

[wp]

Another possibility for very large/small values is to switch to exponential notation above/below some given limit:

Code: Pascal  [Select][+][-]

1. program Project1;
2. uses
3.   SysUtils, Math;
4.  
5.   { Formats number to "nice" string with at most Decs decimal places.
6.     If the absolute value of number is >= Limit or <= 1.0/Limit then exponential
7.     notation is used. }
8.   function NumberToStr(AValue: Double; Decs: Integer = 6; Limit: Double = 1E6): String;
9.   var
10.     fmt: String;
11.   begin
12.     if (abs(AValue) >= Limit) then
13.       fmt := '%.*e'
14.     else
15.     if (abs(AValue) <= 1.0/Limit) then
16.       fmt := '%.*e'
17.     else
18.       fmt := '%.*g';
19.     Result := Format(fmt, [Decs, AValue]);
20.   end;
21.  
22.   procedure Test(x: Double);
23.   begin
24.     WriteLn('NumberToStr(', x, ') = ', NumberToStr(x));
25.   end;
26.  
27. var
28.   x: Double;
29. begin
30.   Test(0.5);
31.   Test(1.0/3);
32.   Test(1E-10);
33.   Test(-1E-10);
34.   Test(1.23456789E15);
35.   Test(1.23456789E15);
36.   ReadLn;
37.  
38. end.

[Tommi]

Much, much simpler:

Code: Pascal  [Select][+][-]

1. begin
2.   writeln(32.47875-32:1:5);// write/writeln has format specifiers....
3. end.

complete program.

Thank you, but I am using a GUI, so writeln isn't an option.

If your calculator always needs to display only numbers in a "medium" range, than a fixed number of decimals might be a satisfying solution. But if it should be able to display very big results, which have e.g. 15 digits (or more) left of the decimal dot or to display very small results like 0.000000000012345, then I think a fixed number of decimals would not be a good solution.

What I did for such cases: I created a formatting function, where I can limit the total number of displayed digits and can limit the maximum displayed digits right of the decimal dot:

Code: Pascal  [Select][+][-]

1. function int_range(x,min,max: integer): integer;
2.    {forces 'x' into the range of [min..max]}
3.    begin
4.    if x < min then exit(min) else
5.       if x > max then exit(max) else exit(x);
6.    end;
7.  
8. function digits_left(d: double): integer;
9.    {returns the number of digits of 'd' which are left to the decimal dot}
10.    var s: string;
11.    begin
12.    str(abs(d):0:38, s);
13.    exit(pos('.',s));
14.    end;
15.  
16. function double_to_total_digits(d: double; digits,decimals: integer): string;
17.    {formats a double to a string.
18.     I: digits: max. total number of digits (left and right of the decimal dot)
19.        decimals: max. number of digits right of the decimal dot}
20.    var s: string;
21.    begin
22.    str(d:0:int_range(digits-digits_left(d), 0,decimals), s);
23.    exit(s);
24.    end;

The benefit is: if you want, you can show big results with less decimals and small results with more decimals. For 'double' I would recommend to use a total of 16 digits.

Code: Pascal  [Select][+][-]

1. begin
2. writeln(double_to_total_digits(1234567890123 + 2/3, 16,16));
3. writeln(double_to_total_digits((1234567890123 + 2/3) * 1E-20, 16,16));
4. end.

Code: Text  [Select][+][-]

1. 1234567890123.667  // gives only 3 digits right of the decimal dot
2. 0.000000012345679  // gives 15 digits right of the decimal dot

I tested your option and it seems to work, but I did a simple change:

Code: [Select]

function double_to_total_digits(d: double; digits,decimals: integer): string;
   {formats a double to a string.
    I: digits: max. total number of digits (left and right of the decimal dot)
       decimals: max. number of digits right of the decimal dot}
var
  s: string;
  a:integer;
begin
 str(d:0:int_range(digits-digits_left(d), 0,decimals), s);
 a:=pos(decimalSeparator,s);
 while (a>0)and((s[length(s)]='0')or(s[length(s)]=decimalSeparator)) do begin s:=copy(s,1,length(s)-1);a:=pos(decimalSeparator,s);end;
 exit(s);
end;


[creaothceann]

The exact number 0.47875 simply does not "exist" in the computer.

In most calculations, ultimate accuracy as offered by big-number libraries is not needed (and even they cannot express the result of 1/3 in an "exact" way because this would require an infinite number of decimal places).

The TFraction type can represent these numbers.

[Thaddy]

There is no need for that in general terms:
The writeln formatting is enough and does not need sysutils - for the expensive format() - nor math.
Why over-complicate things for a basic calculator?

[wp]

There is no need for that in general terms:
The writeln formatting is enough and does not need sysutils - for the expensive format() - nor math.
Why over-complicate things for a basic calculator?

No over-complication. As far as I understand, the TS wants to write a calculator GUI application. Here you can forget WriteLn, and SysUtils is used anyway. The WriteLn's were used in my sample code only for a short demonstration.

[Thaddy]

Thank you, but I am using a GUI, so writeln isn't an option.

Yes it is: we also support writestr. I could have shown my code for a GUI app like so

Code: Pascal  [Select][+][-]

1. procedure TForm1.Button1Click(Sender: TObject);
2. var s:string;
3. begin
4.     writestr(s,32.47875-32:1:5);// write/writeln has format specifiers....
5.     Edit1.text :=  s;
6. end;

Do not fall into the overcomplicating traps provided if the language itself has better means. Anyway, using wp's suggestion is probably best. Lazarus draws in sysutils anyway, so use format() if you don't like writestr.

[srvaldez]

hello Tommi
some time ago I experimented with my own Arbitrary precision decimal floating point arithmetic, for a simple calculator it could be useful
the code is just one file with the test code at the end, no units but a simple file to play with
near the top of file the precision is set in terms of dwords, 8 digits per dword
<edit>
forgot to mention that if you want to use the trig functions then you must assign Pi to fppi like: fppi:=pi_chudnovsky_bs;
and if you want to use logarithms you must assign ln(2) to fpln2 like: fpln2:=calcln2;