[WP34s] Regularized incomplete Beta function

05112014, 01:23 PM
Post: #44




RE: [WP34s] Regularized incomplete Beta function
(05102014 12:03 AM)Paul Dale Wrote: The function is decNumberGammap in decn.c line 2164. Thank you. In my browser the line numbers on the left do not align with the code, but I see where the function code and the series resp. CF routines start. The CF branch uses a modified Lentz algorithm where the value for the "tiny" element is set to 1E37. This is a common value for single precision where the working range ends right below this. The examples in the NR book use a similar value since they obviously assume single precision as well. However, in our case I feel this value should be more like 1E6100. What do you think? Quote:I've fixed the zero return to give a best guess return instead and increased the iteration counts which will help but won't avoid this issue. I do not have a solution yet. On the other hand I found some improvements for the ChiĀ² quantile. The new code is running quite fast and it even handles cases where a simple Newton solver would fail. But it still needs some adjustments for high dof values with probabilities close to zero. Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)