Floating point bug in Canon F-789SGA
11-30-2014, 07:32 PM
Post: #1
 Peter Van Roy Junior Member Posts: 21 Joined: Jan 2014
Floating point bug in Canon F-789SGA
I recently bought a Canon F-789SGA calculator because its specification intriguingly mentions that it has 18 digit precision. Tests show that it actually has 20 digit internal precision, and transcendental functions actually achieve this precision. That is the nice part.

Unfortunately, the calculator has bugs in its floating point operations. It took me a while to figure out what was going on, since there are several bugs that interact in unobvious ways. The main bug is an error in floating point addition. When doing a floating point addition A+B, if the difference in exponents between the two numbers A and B is 16, 17, 18, or 19, and if the lower four digits of the larger number are 0000, then the smaller number is not added in. That is, the lower four digits of the result stay 0000. For example, doing the addition 1 + 1E-16 gives exactly 1 instead of 1.0000000000000001. Floating point subtraction has a similar bug. There are other irregularities in the floating point operations, in particular, the value of the last (20th) digit also plays a role.

My guess to the origin of this bug is that the calculator is an improved version of a Casio calculator with 16 digit precision. Possibly the internal precision was increased without proper testing.

Has anyone else found similar bugs in these or other recent calculators?
 « Next Oldest | Next Newest »

 Messages In This Thread Floating point bug in Canon F-789SGA - Peter Van Roy - 11-30-2014 07:32 PM RE: Floating point bug in Canon F-789SGA - Steve Simpkin - 11-30-2014, 08:36 PM RE: Floating point bug in Canon F-789SGA - Peter Van Roy - 12-01-2014, 10:20 AM RE: Floating point bug in Canon F-789SGA - Gerald H - 12-05-2014, 02:20 PM RE: Floating point bug in Canon F-789SGA - Peter Van Roy - 12-05-2014, 09:39 PM RE: Floating point bug in Canon F-789SGA - Gerald H - 12-06-2014, 06:37 AM RE: Floating point bug in Canon F-789SGA - Peter Van Roy - 12-09-2014, 07:38 PM RE: Floating point bug in Canon F-789SGA - Gerald H - 12-10-2014, 06:35 AM RE: Floating point bug in Canon F-789SGA - Peter Van Roy - 12-13-2014, 06:17 PM RE: Floating point bug in Canon F-789SGA - Gerald H - 12-14-2014, 06:43 AM RE: Floating point bug in Canon F-789SGA - daschel - 02-04-2018, 04:01 AM RE: Floating point bug in Canon F-789SGA - Albert Chan - 07-17-2018, 12:55 AM RE: Floating point bug in Canon F-789SGA - Tugdual - 07-17-2018, 04:08 PM RE: Floating point bug in Canon F-789SGA - Albert Chan - 07-17-2018, 05:27 PM

User(s) browsing this thread: 1 Guest(s)