WP 31S/34S: H.MS+/H.MS- anomaly?

[Paul Dale]

(07-20-2015 06:09 AM)Dieter Wrote:  I see that rounding is a problem with certain H.MS operations, but in this case 18 minutes are exactly 0.3 hours, and this minus 1 hour is exactly –0.7 hours = –42 minutes. So I wonder where a roundoff error may occur.

The conversion of 18 minutes came out as 0.3000....0001 in the internal reals. I didn't check where the error came from but I suspect the multiplication by 1/60 is the culprit. This was chosen because it is faster than division by 60 but it will round differently. Of course, I could be completely wrong here.


- Pauli