The Museum of HP Calculators

HP Forum Archive 14

[ Return to Index | Top of Index ]

Precision of HP 30S
Message #1 Posted by Markus Sigg on 18 Nov 2004, 5:27 a.m.

Hi,

I read that the 30S works with binary representations of numbers, whereas most other pocket calculators use decimal numbers. And I read that its precision is equivalent to 24 decimal places.

Does anybody have an explanation for following results?

(a) 1E10 + 0.11 - 1E10 gives 0.11 (b) 1E10 + 0.01 - 1E10 gives 0

Instead of 0.11 in (a) you can use 0.12345678 also and still get the correct result. So what's happening in (b)???

Things like these make it rather difficult to rely on the precision of this calculator or to estimate error bounds.

Markus

      
Re: Precision of HP 30S
Message #2 Posted by hugh steers on 18 Nov 2004, 7:56 a.m.,
in response to message #1 by Markus Sigg

the hp30s and 9g models do, in fact, work in binary. they also, generally get the correct answers. but there are drawbacks to binary requiring the extra internal precision and also some devious value suppression going on.

its the supression that's expressing the behaviour you've seen. i wrote some stuff about working in binary here:

http://www.voidware.com/binarycalculators.htm

            
Re: Precision of HP 30S
Message #3 Posted by Markus Sigg on 18 Nov 2004, 12:53 p.m.,
in response to message #2 by hugh steers

I think a rounding scheme like that makes the 24 internal digits somewhat useless. I don't see a rational justification for supressing x=0.01, x=0.02, and x=0.03 when calculating 1E10+x but keeping all digits of x=0.11 or even of x=0.12345678.

Are you able to explain the mechanism a bit more detailed? It seems that 1E10+0.01 is changed to 1E10, but 1E10+0.11 is kept. What's the exact rule? Suppose the result of an operation is a. What does the calculator do with this number? Your rounding algorithm needs a second number b to check whether a~b. How does it find b? It seems to me that the decimal expansion of the numbers is used here. Doesn't this imply that a decimal number representation is used?

Markus

                  
Re: Precision of HP 30S
Message #4 Posted by hugh steers on 18 Nov 2004, 2:16 p.m.,
in response to message #3 by Markus Sigg

its because 0.11 is roughly ten times bigger than 0.01.

heres how i think it works (really im guessing). it doesnt just use the final answer `a. it compares `a to one of the terms of the input b. this suppression only applies to ADD and SUB at the top level and not to adds and subs used internally.

define ADD(x, y) to be 0 if |(x + y)/x| < eps, where eps = 1e-12, say. and ADD(x,y) to be x + y, otherwise.

define SUB(x, y) to be 0 if |(x - y)/x| < eps, where eps = 1e-12, say. and SUB(x,y) to be x - y, otherwise.

in the above + means the result of the floating point binary add (ie not true real number addition).

so, in your examples. ex1: 1e10 + 0.11 - 1e10 = 0.11 we have SUB(ADD(1e10,0.11), 1e10) = SUB(10000000000.11,1e10) = 0.11 because (10000000000.11-1e10)/10000000000.11 = 1.0999999..e-11 > eps

but, ex2: 1e10 + 0.01 - 1e10 = 0 because SUB(ADD(1e10,0.01),1e10) = SUB(10000000000.01,1e10) = 0, since (10000000000.01-1e10)/10000000000.01 = 9.9999999..e-13 < eps

                        
Re: Precision of HP 30S
Message #5 Posted by Markus Sigg on 21 Nov 2004, 7:10 a.m.,
in response to message #4 by hugh steers

Your algorithm seems to be correct, thanks. Maybe eps=2e-12 is actually used.

My first thought was that certainly instead of |x+y|/|x| they use |x+y|/max(|x|,|y|), because the addition should be commutative. But they don't. So, for the HP 30S, 1e12 + 1 <> 1 + 1e12, and

1e12 + 1 - 1e12 = 0 1 + 1e12 - 1e12 = 1

Horrible! They build a calculator with excellent internal precision and blow it up by a stupid rounding algorithm.


[ Return to Index | Top of Index ]

Go back to the main exhibit hall