Accuracy?
|
06-18-2021, 09:44 AM
(This post was last modified: 06-18-2021 10:13 AM by HansB.)
Post: #1
|
|||
|
|||
Accuracy?
Why does my Prime G2 shows different results with equal inputs? I like to understand how accurate to how many digits Prime calculates, and how many digits will be displayed. It appears I cannot trust the last digit displayed.
For instance (all textbook input, standard number format, results grouped here for better readability): Code: sqrt(3/4) -> 0.866 025 403 784 (cos(30)) |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
Accuracy? - HansB - 06-18-2021 09:44 AM
RE: Accuracy? - Albert Chan - 06-18-2021, 03:26 PM
RE: Accuracy? - Valentin Albillo - 06-18-2021, 04:04 PM
RE: Accuracy? - HansB - 06-18-2021, 05:01 PM
RE: Accuracy? - rampo - 06-18-2021, 05:16 PM
RE: Accuracy? - John Keith - 06-18-2021, 09:51 PM
RE: Accuracy? - Steve Simpkin - 06-18-2021, 06:40 PM
RE: Accuracy? - rprosperi - 06-18-2021, 10:02 PM
RE: Accuracy? - Valentin Albillo - 06-19-2021, 02:47 AM
RE: Accuracy? - rampo - 06-18-2021, 05:13 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 03:07 PM
RE: Accuracy? - Mike Elzinga - 06-19-2021, 05:46 PM
|
User(s) browsing this thread: 1 Guest(s)