Post Reply 
HP Prime complex division problem in CAS
11-30-2020, 04:52 PM
Post: #24
RE: HP Prime complex division problem in CAS
(11-28-2020 01:13 PM)Albert Chan Wrote:  I just noticed s itself might overflowed, if denominator is small.
We should do scaling in steps:
...

Perhaps multi-step scaling is overkill. Limiting s ≤ MAXREAL work too.

s = (MAXREAL/2) / max(0.5, |RE(x)|,|IM(x)|,|RE(y)|,|IM(y)|)
x/y = cdiv_smith(x*s, y*s)

Patching similar idea to previous cdiv Lua code:
Code:
< xr,xi,yr,yi = ldexp(xr,e),ldexp(xi,e),ldexp(yr,e),ldexp(yi,e)

> e = 2 ^ min(e,1023)
> xr,xi,yr,yi = xr*e, xi*e, yr*e, yi*e
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: HP Prime complex division problem in CAS - Albert Chan - 11-30-2020 04:52 PM



User(s) browsing this thread: 1 Guest(s)