[WP34s] Regularized incomplete Beta function

[Dieter]

(05-10-2014 12:03 AM)Paul Dale Wrote:  The function is decNumberGammap in decn.c line 2164.

Thank you. In my browser the line numbers on the left do not align with the code, but I see where the function code and the series resp. CF routines start.

The CF branch uses a modified Lentz algorithm where the value for the "tiny" element is set to 1E-37. This is a common value for single precision where the working range ends right below this. The examples in the NR book use a similar value since they obviously assume single precision as well. However, in our case I feel this value should be more like 1E-6100. What do you think?

Quote:I've fixed the zero return to give a best guess return instead and increased the iteration counts which will help but won't avoid this issue.
...
Now to try to find a more robust algorithm.

I do not have a solution yet. On the other hand I found some improvements for the Chi² quantile. The new code is running quite fast and it even handles cases where a simple Newton solver would fail. But it still needs some adjustments for high dof values with probabilities close to zero.

Dieter