HP41: accuracy of 13-digit routines
|
09-02-2015, 04:45 AM
Post: #2
|
|||
|
|||
RE: HP41: accuracy of 13-digit routines
(09-01-2015 09:23 PM)Dieter Wrote: In other words: do they really provide correctly rounded 13-digit results under all circumstances? Or may the results be truncated instead of rounded? Or maybe even the last digit may be slightly off? This depends a little on the definition of "correctly rounded". You may try 104985/9999 which is: 10.499549954995... Correctly rounded to 13 digits would give us: 10.49954995500 If we'd round this to 10 digits we'd get: 10.49954996 But that's not what we get. Instead we get: 10.49954995 Thus the internal 13-digit result isn't rounded correctly but cut off. As far as I remeber rounding is just adding 500 to the 13-digit mantissa. Thus in the example from above: Code: 10.49954995499 HTH Thomas |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)