HP41: accuracy of 13-digit routines
|
09-02-2015, 06:11 AM
Post: #3
|
|||
|
|||
RE: HP41: accuracy of 13-digit routines
Thomas is correct. An example that's easier to key in is: \(\sqrt {5} \), which is equal to 2.236067977 499 7896964..., which the HP-41 says is 2.236067977, which shows that it did NOT round to the 13th digit, but truncated at the 13th digit, then rounded THAT to 10 digits.
This "truncate, then round" procedure was not only used by the old 10-digit models, but is still used by the newer 12-digit HP calculators, which calculate internally to 15 *truncated* digits, then round that result to 12 digits. <0|ɸ|0> -Joe- |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)