Post Reply 
06-18-2021, 05:01 PM (This post was last modified: 06-18-2021 06:19 PM by HansB.)
Post: #4
RE: Accuracy?
(06-18-2021 04:04 PM)Valentin Albillo Wrote:  [....]
To add insult to injury, that the (for example) HP-71B computes like this:


.999999999999 (12 digits)

despite doing computations internally with 15-digit accuracy is simply unforgivable. Giving the final result to 12 digits, disregarding the 3 extra digits, is bad enough, but also converting to 12-digit each *intermediate* result (which the user never sees, it's all internal, so no need to convert it to 12-digit), thus losing accuracy wholesale (imagine a longer expression, using functions) is simply unforgivably wasteful and wholly uncalled for.

The proper way to do it is obvious: compute every intermediate result to 15 digits and deliver the final result to 12. That way 1/3+1/3+1/3 gets evaluated internally as .999999999999999 (15 digits) and then returned as exactly 1 (12 digits). Doing otherwise is a bad case of "Kahan"-itis.

I agree. HP Prime does the same
1/3+1/3+1/3 -> .999999999999 (12 digits)
(unless you use CAS view).

So does the Prime work internally to 15 digits, but rounds every intermediate and the end result to 12 digits? Why?
So we get accumulating errors in the twelfth digit, and, when several computations are involved, possibly also in the eleventh and perhaps even the tenth digit.
Estimating a possible error would be like:
+-1 in the last (twelfth) digit, multiplied by the number of calculation steps. Something to bear in mind, especially if many calculating steps are involved, as can happen in a program loop.
Find all posts by this user
Quote this message in a reply
Post Reply 

Messages In This Thread
Accuracy? - HansB - 06-18-2021, 09:44 AM
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021 05:01 PM
RE: Accuracy? - rampo - 06-18-2021, 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM

User(s) browsing this thread: 1 Guest(s)