[WP34S] Inverse t CDF throws "Domain Error" for probabilities close to 0.5 Message #1 Posted by Les Wright on 17 Apr 2012, 7:41 p.m.
In double precision mode, the following ties up the calculator for a painfully long period of time, and ultimately displays "Domain Error":
t^(-1)(.5001, 5), i.e. the t-quantile for the probability given with 5 df
However, the following equivalent problem readily returns the correct result to 34 digits:
sqrt(F^(-1)(.0002, 1,5)) ~= 2.6343e-4
The calculator will work a long time yet return the result in single precision, but if the argument is much closer to 0.5, say 0.500000001, the Domain Error gets thrown in single precision too when one attempts to compute the t-quantile directly.
This occurs with the most recent version of the firmware (2810).
Oddly, it does NOT occur with Qt emulator on my Mac, which reports a firmware version of 2783.
Could it be that some of those changes I instigated regarding the inverse F CDF ended up breaking the inverse t CDF for probabilities near 0.5 (where the quantiles would be close to zero)?
Hope this was worth reporting.
Les
Edited: 17 Apr 2012, 7:50 p.m.
|