### Bookstore

 Computer Math and Games in Pascal Lazarus, the complete guide (only a few left)

### Author Topic: Precision of COMP  (Read 1297 times)

#### Elap

• Jr. Member
• Posts: 52
##### Precision of COMP
« on: February 22, 2016, 11:59:43 am »
When I run the program below in Turbo Pascal, I get the fifth power as 30973432405238707, but in FP 2.6.4 I get 30973432405238700.

In general I don't seem to get the number of significant digits that I would expect for large numbers like these.

Have you any idea why this discrepancy occurs?  I am using Windows 7 Professional and use the console IDE.

Code: Pascal  [Select]
1. {\$R+}
2. PROGRAM BUG3;
3.
4. USES
5.     CRT;
6.
7. VAR
8.     I_comp:      COMP;
9.     I_comp5:     COMP;
10.
11. BEGIN
12.     CLRSCR;
13.
14.     I_comp:=1987;
15.     I_comp5:=I_comp*I_comp*I_comp*I_comp*I_comp;
16.
17.     WRITELN(I_comp:4:0,' to the fifth power is ',I_comp5:17:0);
19. END.
20.

#### marcov

• Global Moderator
• Hero Member
• Posts: 5338
##### Re: Precision of COMP
« Reply #1 on: February 22, 2016, 12:14:47 pm »
I use FPC 3.0 and 3.1.1 both show  *707, the value according to calculator.

#### BeniBela

• Hero Member
• Posts: 547
##### Re: Precision of COMP
« Reply #2 on: February 22, 2016, 12:51:59 pm »
Floating point conversions were completely broken in 2.6.x

Still not working optimally in 3.x, but now it is too precise

#### Elap

• Jr. Member
• Posts: 52
##### Re: Precision of COMP
« Reply #3 on: February 22, 2016, 01:06:17 pm »
Thank you for that.

I'm using Qword which seems to be fine.

#### BeniBela

• Hero Member
• Posts: 547
##### Re: Precision of COMP
« Reply #4 on: February 22, 2016, 01:07:56 pm »
That becomes buggy if you do range checked calculations over high(int64) ...