Post Reply 
[WP34s] Regularized incomplete Beta function
05-11-2014, 01:23 PM
Post: #44
RE: [WP34s] Regularized incomplete Beta function
(05-10-2014 12:03 AM)Paul Dale Wrote:  The function is decNumberGammap in decn.c line 2164.

Thank you. In my browser the line numbers on the left do not align with the code, but I see where the function code and the series resp. CF routines start.

The CF branch uses a modified Lentz algorithm where the value for the "tiny" element is set to 1E-37. This is a common value for single precision where the working range ends right below this. The examples in the NR book use a similar value since they obviously assume single precision as well. However, in our case I feel this value should be more like 1E-6100. What do you think?

Quote:I've fixed the zero return to give a best guess return instead and increased the iteration counts which will help but won't avoid this issue.
Now to try to find a more robust algorithm.

I do not have a solution yet. On the other hand I found some improvements for the ChiĀ² quantile. The new code is running quite fast and it even handles cases where a simple Newton solver would fail. But it still needs some adjustments for high dof values with probabilities close to zero.

Find all posts by this user
Quote this message in a reply
Post Reply 

Messages In This Thread
RE: [WP34s] Regularized incomplete Beta function - Dieter - 05-11-2014 01:23 PM

User(s) browsing this thread: 1 Guest(s)