Post Reply 
WP34s: Domain Error in RMDR and MOD
10-21-2015, 12:13 PM (This post was last modified: 10-21-2015 12:15 PM by Dieter.)
Post: #1
WP34s: Domain Error in RMDR and MOD
I just noticed a strange behaviour of the RMDR and MOD functions. For large arguments they may return a "Domain Error". Some further investigation showed that these functions seem to work for y < 1E820 · x, and as soon as y gets larger the error occurs:

Code:
[DBLON]

[EEX] 820 [ENTER] 360 [x]   =>   3,6 E+822
360 [RMDR]                  =>   Domain Error
    [MOD]                   =>   Domain Error

[x<>y] [ENTER] [ULP] [-]    =>   3,599999999999999999999999999999999 E+822
360 [RMDR]                  =>   80

Since the arguments are out of range in single precision mode, all this happens only after switching to double precision.
Any idea what's going on here?

Dieter
Find all posts by this user
Quote this message in a reply
10-21-2015, 09:39 PM
Post: #2
RE: WP34s: Domain Error in RMDR and MOD
The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails. I didn't go as far as supporting double precision -- there isn't sufficient RAM to do this directly.

Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother Smile


- Pauli
Find all posts by this user
Quote this message in a reply
10-22-2015, 01:00 PM
Post: #3
RE: WP34s: Domain Error in RMDR and MOD
(10-21-2015 09:39 PM)Paul Dale Wrote:  The decNumber mod function isn't efficient and requires enough digits to represent the source numbers and the result (i.e. 2*384 + 16). I increase the number of carried digits for these operations to a little above this so that single precision never fails.

Wait... does this mean that RMDR and MOD internally use 400+ digits?

(10-21-2015 09:39 PM)Paul Dale Wrote:  Making double precision inputs work across the entire range will require a different algorithm and since we don't guarantee any results in double precision I didn't bother Smile

OK. Maybe a table with such restrictions would be helpful. I am also thinking of the distributions, Gamma and Beta with extreme cases of their parameters.

Dieter
Find all posts by this user
Quote this message in a reply
10-22-2015, 09:57 PM
Post: #4
RE: WP34s: Domain Error in RMDR and MOD
Mod uses 450 digits internally.
Modulo reduction by two pi for the trig functions uses 820.

So in double precision, neither is adequate to support the entire range.


Pauli
Find all posts by this user
Quote this message in a reply
Post Reply 




User(s) browsing this thread: 1 Guest(s)