Forum > FV/Textmode IDE

Precision of COMP

(1/1)

Elap:
When I run the program below in Turbo Pascal, I get the fifth power as 30973432405238707, but in FP 2.6.4 I get 30973432405238700.

In general I don't seem to get the number of significant digits that I would expect for large numbers like these.

Have you any idea why this discrepancy occurs?  I am using Windows 7 Professional and use the console IDE.


--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---{$R+}PROGRAM BUG3; USES    CRT; VAR    I_comp:      COMP;    I_comp5:     COMP; BEGIN    CLRSCR;     I_comp:=1987;    I_comp5:=I_comp*I_comp*I_comp*I_comp*I_comp;     WRITELN(I_comp:4:0,' to the fifth power is ',I_comp5:17:0);    READLN;END. 

marcov:
I use FPC 3.0 and 3.1.1 both show  *707, the value according to calculator.

BeniBela:
Floating point conversions were completely broken in 2.6.x

Still not working optimally in 3.x, but now it is too precise

Elap:
Thank you for that.

I'm using Qword which seems to be fine.

BeniBela:
That becomes buggy if you do range checked calculations over high(int64) ...

Navigation

[0] Message Index