Post Reply 
Accuracy?
06-18-2021, 05:16 PM
Post: #6
RE: Accuracy?
[/quote]

I agree. HP Prime does the same
1/3+1/3+1/3 -> .999999999999 (12 digits)
(unless you use CAS view).

So does the Prime work internally to 15 digits, but rounds every intermediate and the end result to 12 digits? So we get accumulating errors in the twelfth digit, and this twelfth digit cannot be trusted?
[/quote]

Interesting, in the Python console
1/3+1/3+1/3
gives 1.0
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Accuracy? - HansB - 06-18-2021, 09:44 AM
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021, 05:01 PM
RE: Accuracy? - rampo - 06-18-2021 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM



User(s) browsing this thread: 1 Guest(s)