Accuracy?
|
06-18-2021, 05:01 PM
(This post was last modified: 06-18-2021 06:19 PM by HansB.)
Post: #4
|
|||
|
|||
RE: Accuracy?
(06-18-2021 04:04 PM)Valentin Albillo Wrote: [....] I agree. HP Prime does the same 1/3+1/3+1/3 -> .999999999999 (12 digits) (unless you use CAS view). So does the Prime work internally to 15 digits, but rounds every intermediate and the end result to 12 digits? Why? So we get accumulating errors in the twelfth digit, and, when several computations are involved, possibly also in the eleventh and perhaps even the tenth digit. Estimating a possible error would be like: +-1 in the last (twelfth) digit, multiplied by the number of calculation steps. Something to bear in mind, especially if many calculating steps are involved, as can happen in a program loop. |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021 05:01 PM
RE: Accuracy? - rampo - 06-18-2021, 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM
|
User(s) browsing this thread: 1 Guest(s)