Post Reply 
Accuracy and the power function
02-21-2014, 07:11 AM
Post: #13
RE: Accuracy and the power function
Hi Dieter,
no it doesn't. That's where the Sticky Bit comes in: the CPU works with 15 digits only, aligning the mantissa's (in case of addition), or repeatedly add-and-shift in case of multiplication. In both cases it will keep track of the digits lost to the right, just to know whether they're all zero or not. That's why it's called a 'Sticky Bit': once you shifted a non-zero digit to the right, it is set, and remains set.
So, in your example:
5.02000000124e15 + 5009 (exact result: 5.020000001245009e15) would be truncated to 5.02000000124500, BUT the Sticky Bit would be set, as the '9' we lost is non-zero.
And the routine that rounds 15 digits to 12 will round a trailing '500' up when it is set, and round to even when it is clear.
That's why all 12-digit HPs do get the correctly rounded answer for this example (or any other), yet they all truncate their 15-digit result.

Find all posts by this user
Quote this message in a reply
Post Reply 

Messages In This Thread
Accuracy and the power function - Dieter - 02-16-2014, 02:43 PM
RE: Accuracy and the power function - Werner - 02-21-2014 07:11 AM

User(s) browsing this thread: 1 Guest(s)