Accuracy?
|
06-18-2021, 05:16 PM
Post: #6
|
|||
|
|||
RE: Accuracy?
[/quote]
I agree. HP Prime does the same 1/3+1/3+1/3 -> .999999999999 (12 digits) (unless you use CAS view). So does the Prime work internally to 15 digits, but rounds every intermediate and the end result to 12 digits? So we get accumulating errors in the twelfth digit, and this twelfth digit cannot be trusted? [/quote] Interesting, in the Python console 1/3+1/3+1/3 gives 1.0 |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021, 05:01 PM
RE: Accuracy? - rampo - 06-18-2021 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM
|
User(s) browsing this thread: 1 Guest(s)