Forum > Beginners
Converting variables and subtracting them
andhunter:
Hi everyone!
Can you please tell me why I get wrong number when subtracting? Here is an example 5.05 - 5 = 0.0499999999999998
And this always happens when the integer parts of the numbers are the same (10 and 10.05, 20 and 20.05, etc.). As far as I understand, this can be bypassed using StrToFloatF . But I would like to know what causes this result.
--- Code: Pascal [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---unit Unit1; {$mode objfpc}{$H+} interface uses Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls; type { TForm1 } TForm1 = class(TForm) Button1: TButton; Edit1: TEdit; Edit2: TEdit; Edit3: TEdit; Label1: TLabel; procedure Button1Click(Sender: TObject); private public end; var Form1: TForm1; a,b,c:extended;implementation {$R *.lfm} { TForm1 } procedure TForm1.Button1Click(Sender: TObject);begin a:= StrToFloat(Edit1.Text); b:= StrToFloat(Edit2.Text); c:= a-b; // 5,05 - 5 = 0,0499999999999998 Label1.Caption:= floattostr(c); end; end.
KodeZwerg:
Because Floats are no natural number. Even a float of 10 is something like 10.00000000000952436593264593214
KodeZwerg:
Try something like:
--- Code: Pascal [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---Label1.Caption := FloatToStrF(c, ffFixed, 2, 2);
andhunter:
--- Quote from: KodeZwerg on January 24, 2023, 04:15:30 pm ---Try something like:
--- Code: Pascal [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---Label1.Caption := FloatToStrF(c, ffFixed, 2, 2);
--- End quote ---
Thank you for the explanation. The Windows calculator also apparently uses a similar rounding method.
Warfley:
--- Quote from: KodeZwerg on January 24, 2023, 04:08:43 pm ---Because Floats are no natural number. Even a float of 10 is something like 10.00000000000952436593264593214
--- End quote ---
I don't know how you get this number, but thats not true. Floats can display any whole number thats between 0 and 2^m (where m is the number of mantissa bits) correctly. The reason for this is that an IEEE 754 float is of the form 1.mantissa *2^exponent. Where the mantissa can be any bitstring. Meaning for any number whose bit representation fits the mantissa, an exponent can be chosen, such that the whole mantissa is shifted over the decimal separator.
What cannot be fully realized are fractions. Because the of the form 1.mantissa, with mantissa being a bitstring, only combinations of fractions with exponent 2 can be expressed. For example 0.5 (1/2), 0.25 (1/4), 0.75 (1/2+1/4), etc. can be written this way.
We humans usually use the decimal system, which is base 10. As 10 is a prime combination of 2 and 5, this means that every fraction of an exponent of 2 can also be written as a decimal number. But as some decimal numbers contain fractions of base 5, not all can be dispalyed as binary fractions.
The same phenomena can be seen when looking at our decimal system and fractions of neither exponent 2 or 5. 1/3 for example cannot be represented as a decimal number, but is 0.333333... So at some point it must be cut of, for example to 0.333, but now this number is 0.0003333... = 1/3000 different from the original number (i.e. we now display 1/3-1/3000 rather than just 1/3). And the exact same thing happens when we try to convert decimal fractions to binary and vice versa.
For example, the 5.05 is 5 + 0.05. Again whole numbers can be fully represented, but looking at the fraction 0.05 = 1/2 * 1/5 * 1/2. As you can see it contains a fraction of base 5 (1/5), so it cannot be represented as a binary fraction.
This is why it is just barely wrong in the computer, and when outputting without any rounding, it will show you all the digits up to the error bit in the end.
Navigation
[0] Message Index
[#] Next page