 The Museum of HP Calculators

HP Articles Forum

HP-17BII precision: Epsilon

Posted by W. B. Maguire II on 24 Sept 2001, 4:33 p.m.

## Calculating Epsilon on the HP-17BII

I wrote a program to compute 'Epsilon' which---in computational context---means the smallest number (in the computer) that may be added to one and yield a result that is greater than one. It is a measure of the precision that the computer uses to represent numbers (really the number of bits used for the mantissa).

I first wrote a traditional "Epsilon" equation for the HP-17BII, but I realized that the number storage was not the standard binary floating point, but rather some form of binary-coded-decimal. As a result, the original "Epsilon" program yielded an incorrect result. The general algorithm may be used on any machine that stores floating point numbers in an IEEE-like format. So, I re-wrote the equation for a binary-coded-decimal machine. The new equation yields the correct result for the HP-17BII; the Epsilon is 5.00000000001e-12 (twelve digits in the mantissa).

Binary Floating-Point Epsilon algorithm---INCORRECT for HP-17BII:

```Epsilon = 1;
while ((1 + Epsilon) > 1) {
Epsilon /= 2;
}
Epsilon *= 2;
```

Below is the HP-17BII equation. Please note that in the listing below "SIGMA" means the sigma symbol:
In the ALPHA menu:
[WXYZ][OTHER][MORE],
then the second button

Binary-Coded-Decimal equation for the HP-17BII:

```EPSILON: 0*L(E:1)*
SIGMA(N:1:25:1:(
SIGMA(I:1:9:1:
IF(1+L(X:(G(E)-I*10^(-N)))>1:
L(NEWE:G(X))
:
0
)
)
+L(OLDE:G(E))
+L(E:G(NEWE))
)
)
+
IF(G(OLDE)>G(E):
0
:
G(E)
)
=EPSILON
```

The extra code concerning the "OLDE" variable is just to make sure that we have looped far enough. If Epsilon comes back as zero, then we need to search more powers of ten in the "N" loop (i.e. increasing the value of 25 here).