(12C) Y^X for Large Numbers

[Dieter]

(01-30-2018 06:03 PM)Carsen Wrote:  And I realized the error that this program produces but I didn't think it was in correlation to the amount of digits with the exponent. I have a question though. Does this error come from the 12C being unable to hold as many decimal places (internally and externally) because of how big the integer is?

It's quite simple. The program calulates the base-10 log of the desired power. This can be split into an integer part and an fractional part. For 110^125 this log is 255,1740857. So the result is 10^255 * 10^0,1740857 = 10^255 * 1,493...

Since the exponent 255 occupies three digits there are only seven digits left for the fractional part from which the mantissa is calculated. And since there always is a potential error of 1 unit in the last place (compare the results of your and my program) the mantissa can be off by a factor of 10^1E–7 = 1,000000023 (≈1+ln10·1E–7) which means a relative error of 2,3E–7. This is equivalent to an absolute error between 1*2,3E–7 and 9,999...*2,3E–7 = 2,3E–6 by which the mantissa can be off. This again means six or five valid decimals, or seven or six significant digits.

Try it: The calculated base-10 log of 120^555 is 1153,945591. The actual value is 1153,9455915564... so the log here even is almost correct to 10 digits. But 10^0,945591 is 8,822486450 while only 10^0,9455915564 yields the correct mantissa 8,822497753. The calculated result is only correct to five significant digits: 8,8225.

So the reason for the limited accuracy is simply the number of remaining digits for the fractional part from which the mantissa is calculated. The more digits the integer part (= exponent) has the less remain for the fractional part, i.e. for the mantissa.

Rule of thumb: if the exponent has n digits you should view the mantissa in FIX 9–n (if the mantissa is ≤ 2) or even in FIX 8–n (if it is larger). This limits the error to 1 unit in the last displayed digit. For the last example (8,822...E+1153) this means FIX 8–4 = FIX 4. Likewise the result of the first example 110^125 = 1,493...E+255 can be viewed in FIX 9–3 = FIX 6.

Dieter