A quick precision test

[Paul Dale]

This test is not testing precision. At least when used as you seem to have.


1.0000001 has eight digits.
1.0000001^2 has sixteen digits (x^2 once).
1.0000001^4 has thirty two digits (x^2 twice).

Doing x^2 twenty seven times is raising the original number to the power 2^27 = 134,217,728. This has 8 * 134,217,728 = 1,073,741,824 decimal digits. That is a thousand million digits there. No calculator known can deal with that many, many desktop systems would fail or at best struggle.

Since you are doing each squaring separately, there is a rounding step each time. I'm confident these roundings explain the differing results. The 34S has guaranteed correctly rounded multiplication in single precision so it is giving the precise correct answer for a sixteen digit decimal device.


- Pauli