Floating point bug in Canon F-789SGA

[Peter Van Roy]

(12-05-2014 02:20 PM)Gerald H Wrote:  i'm not sure it's a bug.

Handbook says:

"Precision +/- 1 at the 10th digit for a single calculation
+/- 1 at the least significant for exponential display" (page 10)

which I don't really understand, but could imply you are requiring accuracy above specifications.

The specification says clearly that internal precision is 18 digits (see page 11 of F-789SGA Scientific Calculator User Instructions). It is quite common for calculators to have higher internal precision than what they display. My observations show that floating point addition has a bug at the 17th and 18th digits.