Search
Member List
Calendar
Help
|
Casio rounding
|
|
11-14-2018, 04:28 PM
Post: #5
|
|||
|
|||
RE: Casio rounding
(11-14-2018 01:59 PM)Eddie W. Shore Wrote: Interesting. The rounding scheme seems to be has been arbitrarily chosen instead of going with the traditional rule of looking at the last digit and rounding up when it is 5 or more. I think it's just the usual trick that some calculators do to "correct" small round-off errors instead of leaving them alone. However, these "corrections" can make the result worse. For example, 920/99 would produce an internal answer of 9.29292929292 or 9.29292929293 which the FX-602P would round off internally to 9.29292929300 before any subsequent operations were performed on it. (I haven't tested this on an actual FX-602P.) — Ian Abbott |
|||
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
Casio rounding - Pekis - 11-13-2018, 01:19 PM
RE: Casio rounding - Albert Chan - 11-13-2018, 04:24 PM
RE: Casio rounding - Eddie W. Shore - 11-14-2018, 01:59 PM
RE: Casio rounding - toml_12953 - 11-14-2018, 03:39 PM
RE: Casio rounding - Eddie W. Shore - 11-19-2018, 01:46 PM
RE: Casio rounding - toml_12953 - 11-19-2018, 03:05 PM
RE: Casio rounding - ijabbott - 11-14-2018 04:28 PM
|
User(s) browsing this thread: 1 Guest(s)