Accuracy and the power function

02202014, 05:17 PM
Post: #12




RE: Accuracy and the power function
(02192014 01:07 PM)Werner Wrote: Truncation (and keeping track of lost nonzero digits with the Sticky Bit) in the 15digit routines is not only common practice, it's a necessity if you want to be able to produce a correctly rounded 12digit result for simple calculations. Thank you for this example. Here's another one: 5.02000000124e15 + 5009 (exact result: 5.020000001245009e15) would be rounded to 15 digits as 5.02000000124501e15. After rounding this to 12 digits (towards odd or even) the result is 5.02000000125e15. Which is the correct 12digit value. 5.02000000124e15 + 5009 (exact result: 5.020000001245009e15) would be truncated after 15 digits to 5.02000000124500e15. After rounding this to 12 digits (towards even) the result is 5.02000000124e15. Which is 1 ULP low. So, as usual, it depends. ;) Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)