HP PRIME not raising overflow/underflow may cause critical errors

[Albert Chan]

(03-18-2021 08:35 PM)robve Wrote:  This means that arithmetic can be broken at times:

H := G/2
F/G^2==1/4*F/H^2
0
F/G^2==F/G/G
0

Even with IEEE-754, once hit by overflow/underflow, results are questionable.

lua> x = 1e300
lua> x/x/x, x/x^2
1e-300       0

We also have this weird x/x/x ≠ x/x^2 = x/x^2/2 = x/x^2/3 = x/x^3/4 ...

I do like IEEE-754 design that Inf does not turned back to finite value.
(not always, since 1/±Inf underflowed back to ±0.0)

Final result of Inf/NaN implied overflowed intermediate values.

(03-19-2021 04:12 PM)robve Wrote:  Let's see if and how HP PRIME handles overflow in one of its built-in functions:

horner([1,-1,-1,-1,-1,-1,-1,-1,0],1e38)
1E304
horner([1,-1,-1,-1,-1,-1,-1,-1,0],1E39)
+Inf

Apparently horner performs a quick check on the magnitude of the list's leading coefficients to return +Inf.
Quick checks like these are typically overly conservative. The result is actually 1E312:

"horner" is from the CAS side, thus calculating in binary math.
Since 1E312 > 2^1024, horner returned +Inf.

HOME> horner([1,0,-PI*PI], PI)     // to show horner is really doing binary math
6.25277607469ᴇ−13                     // 11/2^44 converted back to 12-digits decimal

(03-20-2021 02:23 PM)robve Wrote:  

(03-20-2021 06:05 AM)Han Wrote:  Context makes a huge difference in mathematics, though. And depending on the context, it may be 1, or 0, or undefined. It is defined in those contexts by choice in order to have a consistent system. Hence the "consensus." The wikipedia link gives plenty of well-reasoned examples of why they can be taken as any of these. Here is another resource:

LOL, I do not see any "consensus" in the resources you point to that \( 0^0=1 \) is a universal truth.

You might want to re-read Han's post again.
The quote actually support your claim that 0^0 is indeterminate.

n = exp(ln(n)/ln(x)*ln(x)) = x^(ln(n)/ln(x))

Taking the limit for any x, we get n back.

x->0: 0^(ln(n)/∞) = 0^0 = n
x->1: 1^(ln(n)/0) = 1^∞ = n