Post Reply 
Casio rounding
11-14-2018, 04:28 PM
Post: #5
RE: Casio rounding
(11-14-2018 01:59 PM)Eddie W. Shore Wrote:  Interesting. The rounding scheme seems to be has been arbitrarily chosen instead of going with the traditional rule of looking at the last digit and rounding up when it is 5 or more.

I think it's just the usual trick that some calculators do to "correct" small round-off errors instead of leaving them alone. However, these "corrections" can make the result worse. For example, 920/99 would produce an internal answer of 9.29292929292 or 9.29292929293 which the FX-602P would round off internally to 9.29292929300 before any subsequent operations were performed on it. (I haven't tested this on an actual FX-602P.)

— Ian Abbott
Find all posts by this user
Quote this message in a reply
Post Reply 

Messages In This Thread
Casio rounding - Pekis - 11-13-2018, 01:19 PM
RE: Casio rounding - Albert Chan - 11-13-2018, 04:24 PM
RE: Casio rounding - Eddie W. Shore - 11-14-2018, 01:59 PM
RE: Casio rounding - toml_12953 - 11-14-2018, 03:39 PM
RE: Casio rounding - Eddie W. Shore - 11-19-2018, 01:46 PM
RE: Casio rounding - toml_12953 - 11-19-2018, 03:05 PM
RE: Casio rounding - ijabbott - 11-14-2018 04:28 PM

User(s) browsing this thread: 1 Guest(s)