HP41: accuracy of 13digit routines

09022015, 06:11 AM
Post: #3




RE: HP41: accuracy of 13digit routines
Thomas is correct. An example that's easier to key in is: \(\sqrt {5} \), which is equal to 2.236067977 499 7896964..., which the HP41 says is 2.236067977, which shows that it did NOT round to the 13th digit, but truncated at the 13th digit, then rounded THAT to 10 digits.
This "truncate, then round" procedure was not only used by the old 10digit models, but is still used by the newer 12digit HP calculators, which calculate internally to 15 *truncated* digits, then round that result to 12 digits. <0ɸ0> Joe 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)