WP34s: Domain Error in RMDR and MOD
10-21-2015, 12:13 PM (This post was last modified: 10-21-2015 12:15 PM by Dieter.)
Post: #1
 Dieter Senior Member Posts: 2,398 Joined: Dec 2013
WP34s: Domain Error in RMDR and MOD
I just noticed a strange behaviour of the RMDR and MOD functions. For large arguments they may return a "Domain Error". Some further investigation showed that these functions seem to work for y < 1E820 · x, and as soon as y gets larger the error occurs:

Code:
[DBLON] [EEX] 820 [ENTER] 360 [x]   =>   3,6 E+822 360 [RMDR]                  =>   Domain Error     [MOD]                   =>   Domain Error [x<>y] [ENTER] [ULP] [-]    =>   3,599999999999999999999999999999999 E+822 360 [RMDR]                  =>   80

Since the arguments are out of range in single precision mode, all this happens only after switching to double precision.
Any idea what's going on here?

Dieter
10-21-2015, 09:39 PM
Post: #2
 Paul Dale Senior Member Posts: 1,467 Joined: Dec 2013
RE: WP34s: Domain Error in RMDR and MOD
The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails. I didn't go as far as supporting double precision -- there isn't sufficient RAM to do this directly.

Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother

- Pauli
10-22-2015, 01:00 PM
Post: #3
 Dieter Senior Member Posts: 2,398 Joined: Dec 2013
RE: WP34s: Domain Error in RMDR and MOD
(10-21-2015 09:39 PM)Paul Dale Wrote:  The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails.

Wait... does this mean that RMDR and MOD internally use 400+ digits?

(10-21-2015 09:39 PM)Paul Dale Wrote:  Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother

OK. Maybe a table with such restrictions would be helpful. I am also thinking of the distributions, Gamma and Beta with extreme cases of their parameters.

Dieter
10-22-2015, 09:57 PM
Post: #4
 Paul Dale Senior Member Posts: 1,467 Joined: Dec 2013
RE: WP34s: Domain Error in RMDR and MOD
Mod uses 450 digits internally.
Modulo reduction by two pi for the trig functions uses 820.

So in double precision, neither is adequate to support the entire range.

Pauli
 « Next Oldest | Next Newest »

User(s) browsing this thread: 1 Guest(s)