Recent

Author Topic: Rounding problem  (Read 11967 times)

Pi

  • Jr. Member
  • **
  • Posts: 75
Rounding problem
« on: March 25, 2012, 08:31:17 am »
I have a function that worked as expected in the older version of Lazarus/Free Pascal, but now in the later version, doesn't work as expected.

The code is supposed to round to 2 decimal places.
eg.
Code: [Select]

var the_nr: real;
begin       

 the_nr:=14.321;

  showmessage('before round '+floattostr(the_nr));
 
  the_nr:= round((the_nr*100.0))/100.0; // This is supposed to round the_nr to 2 decimal places

  showmessage('after round '+floattostr(the_nr)); 
In Lazarus 0.9.30, FPC 2.4.2, Windows Vista, 32 bit, the above code gives: "before round 14.321", "after round 14.32" (correct)

In Lazarus 0.9.30.4, FPC 2.6.0, Windows 7, 64 bit, the above code gives: "before round 14.321", after round 14.3199996948242" (wrong)

 
edit: If I break it down (in Lazarus 0.9.30.4, FPC 2.6.0) to 3 seperate statements, it works.
eg.
Code: [Select]
the_nr:=14.321;

 the_nr:=the_nr*100.0;
  the_nr:=round(the_nr);
  the_nr:=the_nr/100.0;

works (gives 14.32)

but

Code: [Select]
the_nr:=14.321;

the_nr:=round((the_nr*100.0))/100.0;
doesn't work (gives 14.3199996948242)

I'll use the 3 seperate statements for now, but isn't there a problem with the way it it compiles the more compact version?

[in Compilation Options->Parsing,  Syntax mode is set to Object Pascal - default]
« Last Edit: March 25, 2012, 09:26:55 am by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

ludob

  • Hero Member
  • *****
  • Posts: 1173
Re: Rounding problem
« Reply #1 on: March 25, 2012, 11:13:13 am »
What happens if you replace
Code: [Select]
var the_nr: real;with
Code: [Select]
var the_nr: double;?

Real is not a fixed type and can change from processor to processor. I see you changed compiler but also processor.

tomek

  • Jr. Member
  • **
  • Posts: 85
Re: Rounding problem
« Reply #2 on: March 25, 2012, 11:43:59 am »
That is because round return int64.

Code: [Select]
the_nr:= int(round(the_nr*100.0))/100.0;
but I agree that this creates a lot of confusion.

User137

  • Hero Member
  • *****
  • Posts: 1791
    • Nxpascal home
Re: Rounding problem
« Reply #3 on: March 25, 2012, 11:50:42 am »
Float is not a precise type, floattostr() is not a precise printing method either. Use format('%.2f', [the_nr])

I mean when working with floating points, expect to find cases where 0.99999999, 1.0 and 1.00000001 all mean same value.
« Last Edit: March 25, 2012, 11:54:14 am by User137 »

tomek

  • Jr. Member
  • **
  • Posts: 85
Re: Rounding problem
« Reply #4 on: March 25, 2012, 11:54:18 am »
Code: [Select]
var the_nr, the_nr2: real;
begin       

 the_nr:=14.321;

  showmessage('before round '+floattostr(the_nr));
  the_nr2:=round(the_nr*100.0);
  the_nr:= the_nr2/100.0; // This is supposed to round the_nr to 2 decimal places

  showmessage('after round '+floattostr(the_nr))

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #5 on: March 25, 2012, 12:14:40 pm »
What happens if you replace
Code: [Select]
var the_nr: real;with
Code: [Select]
var the_nr: double;?

Real is not a fixed type and can change from processor to processor. I see you changed compiler but also processor.
When I change the_nr from real to double, no change in the results given by the code occurs.

the_nr:=14.321;
the_nr:= round((the_nr*100.0))/100.0;
// makes floattostr(the_nr) 14.3199996948242

and putting it in 3 seperate statements makes it 14.32

I've also just tried changing the_nr to valreal and also tried extended, and they also don't make "the_nr:=round((the_nr*100.0))/100.0" any more accurate.
« Last Edit: March 25, 2012, 01:27:44 pm by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #6 on: March 25, 2012, 12:22:01 pm »
Float is not a precise type, floattostr() is not a precise printing method either. Use format('%.2f', [the_nr])

I mean when working with floating points, expect to find cases where 0.99999999, 1.0 and 1.00000001 all mean same value.
Thanks, I'll use that.  Though using "round((the_nr*100.0))/100.0" gave an accurate result before in a different version of Lazarus/FPC on my other PC, without having to use something other than floattostr.

Couldn't there be a problem with the way the current version of Lazarus/FPC (eg. the compiler/parser) is handling the code "round((the_nr*100.0))/100.0" (since a past Lazarus/FPC version on a different PC was more accurate which meant floattostr could be used then)?
« Last Edit: March 25, 2012, 01:15:44 pm by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #7 on: March 25, 2012, 12:31:36 pm »
Code: [Select]
var the_nr, the_nr2: real;
begin       

 the_nr:=14.321;

  showmessage('before round '+floattostr(the_nr));
  the_nr2:=round(the_nr*100.0);
  the_nr:= the_nr2/100.0; // This is supposed to round the_nr to 2 decimal places

  showmessage('after round '+floattostr(the_nr))
Thanks.  That also gives an accurate result (but by increasing the statements & nr of variables).
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #8 on: March 25, 2012, 12:52:22 pm »
That is because round return int64.

Code: [Select]
the_nr:= int(round(the_nr*100.0))/100.0;
but I agree that this creates a lot of confusion.
Thanks.  That also seems to work.
Though since "round" returns "int64" I don't see why the added "int" was needed.  According to http://www.freepascal.org/docs-html/rtl/system/int.html
Int allows you to "Calculate integer part of floating point value", but above it's being used on the result of "round", which is an int64.  Yet it does give give a different (and also seems to be the correct) result.  Which is confusing.
« Last Edit: March 25, 2012, 01:03:36 pm by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

tomek

  • Jr. Member
  • **
  • Posts: 85
Re: Rounding problem
« Reply #9 on: March 25, 2012, 02:24:06 pm »
Thanks.  That also seems to work.
Though since "round" returns "int64" I don't see why the added "int" was needed.  According to http://www.freepascal.org/docs-html/rtl/system/int.html
Int allows you to "Calculate integer part of floating point value", but above it's being used on the result of "round", which is an int64.  Yet it does give give a different (and also seems to be the correct) result.  Which is confusing.

I give that example to show that int64 is otherwise treated by the 64bit processor.

avra

  • Hero Member
  • *****
  • Posts: 2582
    • Additional info
Re: Rounding problem
« Reply #10 on: March 26, 2012, 10:13:35 am »
Try:
Code: [Select]
the_nr := trunc(the_nr * 100 + 0.5) / 100.0;
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #11 on: March 26, 2012, 02:49:55 pm »
Try:
Code: [Select]
the_nr := trunc(the_nr * 100 + 0.5) / 100.0;
Thanks. 

That works like my original function though, in that, on my Windows Vista PC (32 bit windows) running Lazarus v0.9.30, using your function with the_nr set to 14.321 then converting to string using floattostr gives "14.32" [correct]
Though on my Win 7, 64 bit PC, Lazarus v0.9.30.4, FPC 2.6.0 (like with my original function) the result your function gives after using floattostr is "14.3199996948242"

whereas some of the other methods give a 2 dp (eg 14.32) result after using floattostr on both versions of Lazarus/FPC/on both my PCs, so I think the other methods are more accurate, at least on this PC/Lazarus/FPC version. 

Though if you break your version down into more statements,
eg.
Code: [Select]
the_nr:=14.321;

  the_nr:=(the_nr*100+0.5);
  the_nr:=trunc(the_nr);
  the_nr:=the_nr/100.0;   
It gives the more precise answer (after using floattostr), even on my Win 7 64 bit machine, ie. "14.32" not "14.319999694824".

I would have expected that the compiler should have generated the same code for both versions (the 1 line version of the code, and the 3 line version of it), though that can't be the case, and it seems that on this version of Lazarus/FPC the compiler generates less accurate code (ie. one that makes a less accurate calculation result) in the first version of the code (using your code all in one statement) than the compiler does if the same calculation is written in 3 seperate statements.

Also, I don't think it's just a case of "floattostr" not giving an accurate result (nor only because of inaccuracies with floating point numbers in general), as I've tried it with my own function that converts an extended to a string (and using an extended instead of a real for the_nr - even though I think trunc uses a normal real not extended), and the version of your function using 3 statements instead of 1 also gives a lot more accurate result using that.
« Last Edit: March 26, 2012, 04:30:34 pm by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

Shebuka

  • Sr. Member
  • ****
  • Posts: 429
Re: Rounding problem
« Reply #12 on: March 26, 2012, 04:41:50 pm »
Before making conclusions about 'compiler calculations' try install Lazarus 0.9.30, FPC 2.4.2 on your Windows 7 x64 and try same code from Windows Vista x32.

Pi

  • Jr. Member
  • **
  • Posts: 75
Re: Rounding problem
« Reply #13 on: March 26, 2012, 05:54:30 pm »
Before making conclusions about 'compiler calculations' try install Lazarus 0.9.30, FPC 2.4.2 on your Windows 7 x64 and try same code from Windows Vista x32.
Well I've tried copying the .exe of a program that was compiled using Lazarus 0.9.30 FPC 2.4.2 on Win Vista to my Windows 7 64 Bit PC and in that version, when that program is run on my Win 7 PC, the method shown by Avra above using trunc, all on one line gives 14.32 as the result of the rounding, not the less accurate result.

Though I don't want to go back to a past Lazarus & FPC version on this Win 7 PC as it would cause other things to work incorrectly.  Would there be any conflicts having 2 different Lazarus versions on one PC?

But the difference in compilation I was talking about above in response to avra was mostly about the difference between how the compiler (Laz 0.9.30.4, FPC 2.6.0) compiles
Code: [Select]
the_nr := trunc(the_nr * 100 + 0.5) / 100.0
compared to how it compiles
Code: [Select]
  the_nr:=(the_nr*100+0.5);
  the_nr:=trunc(the_nr);
  the_nr:=the_nr/100.0; 
Surely both above code sections should compile about the same and give a result approx the same (or at least, the top one shouldn't be a lot less accurate than the other way) if using the same compiler and Lazarus/FPC version, PC & settings?

It's not a problem for me though, since I can just use one of the the longer code versions instead of the shorter (less accurate) versions.
« Last Edit: March 26, 2012, 08:00:50 pm by Pi »
Lazarus version 0.9.30.4 + Lazarus 32 bit 1.2.6

KpjComp

  • Hero Member
  • *****
  • Posts: 680
Re: Rounding problem
« Reply #14 on: March 28, 2012, 11:45:39 pm »
Quote
Code: [Select]
the_nr := trunc(the_nr * 100 + 0.5) / 100.0compared to how it compiles
Code: [Select]
  the_nr:=(the_nr*100+0.5);
  the_nr:=trunc(the_nr);
  the_nr:=the_nr/100.0; 
Surely both above code sections should compile about the same and give a result approx the same

Well, there both not the same.
Your multi-liner is more equivalent to ->
Code: [Select]
the_nr := real(trunc(real(the_nr*100+0.5)))/100;
Because each assignment to the_nr is causing a type conversion, that your original one liner wasn't doing.
« Last Edit: March 28, 2012, 11:48:06 pm by KpjComp »

 

TinyPortal © 2005-2018