HP Forum Archive 19

[ Return to Index | Top of Index ]

Curve fitting question (OT) Message #1 Posted by Walter B on 29 July 2009, 9:30 a.m. Hi experts, So far, I thought I know some statistical data analysis, but here's something strange leaving me confused: Fitting a model curve to measured data points, i.e. data with errors > 0 , reduced chi square (RC2) is one method to assess the goodness of fit. Browsing some textbooks and the net, I find just one-sided tests for RC2. I.e. it is said to be a good fit model if RC2 is less than or approximately equal to 1, and a bad fit model if RC2 >> 1. Why isn't this a two-sided test? Reasoning: The model curve hitting all data points right in the middle is very improbable for physical data. So it should be a two-sided test for RC2, excluding very low values of RC2, too. OTOH, fitting is done minimizing the sum of squared distances from all data points to the curve, so zero would be optimum. You see me confused. What's wrong? Help is appreciated. Walter

Re: Curve fitting question (OT) Message #2 Posted by Namir on 29 July 2009, 1:35 p.m., in response to message #1 by Walter B Hi Walter, Most books use R-square and the F statistics to measure goodness of fit. One popular tool is found in the Excel Data Analysis add-in. It generates an ANOVA table that shows the F statistics and its significance. Also, the tool calculates and shows the R-square and its adjusted value. Finally the output also shows the standard error, student-t, and significance of student-t for each regression coefficient (and intercept). I rely on the regression ANOVA table and other statistics displayed by the Excel Data Analysis add-in more than just the Chi-square. If you want to examine the relevance of yoru curve fit I highly recommend Excel's tool and the richness of the output. Namir

Re: Curve fitting question (OT) Message #3 Posted by Walter B on 30 July 2009, 6:02 p.m., in response to message #2 by Namir Hi Namir, Thanks for your friendly advise. I know and use this tool already. It is just that almost every author I see demonstrating simple Chi Square tests uses single sided tests to measure goodness of fit, while the same authors have no problems with double sided t-tests, for example. That's what puzzles me. Maybe a bit academic, but I just want to understand it. Walter

Re: Curve fitting question (OT) Message #4 Posted by Paul Dale on 29 July 2009, 5:45 p.m., in response to message #1 by Walter B Quote: Why isn't this a two-sided test? You're squaring the differences so the closer to zero you get the better the fit. Thus no need for the other side. Or am I missing something in the question? - Pauli

Re: Curve fitting question (OT) Message #5 Posted by Walter B on 29 July 2009, 6:09 p.m., in response to message #4 by Paul Dale Hi Pauli, Quote: Or am I missing something in the question? Maybe. Please check my "reasoning" above and Dave's answer below. IMHO a contradiction remains. Walter Edited: 30 July 2009, 3:03 a.m.

Re: Curve fitting question (OT) Message #6 Posted by Paul Dale on 30 July 2009, 5:09 p.m., in response to message #5 by Walter B Missed that bit for some reason -- probably my own stupidity and haste :-( I've never heard of a test for a too-good fit of the data. That has always been a cause for great rejoicing not concern. However, if you are interested in that why not do another test at the 99.9999% (or whatever) level and reject if that accepts the fit? - Pauli

Re: Curve fitting question (OT) Message #7 Posted by Dave Shaffer (Arizona) on 29 July 2009, 6:02 p.m., in response to message #1 by Walter B Quote: so zero would be optimum. Not really! Reduced chi-square includes the estimated size of your errors, so if you have estimated your errors correctly (sometimes a task in itself for real-world data), the fit value should be near 1. If it is much below one, either your error estimates are not good (too pessimistic), or you are fitting to too many variables or the wrong equation(s) and fooling yourself that you know what is going on!

Re: Curve fitting question (OT) Message #8 Posted by Walter B on 30 July 2009, 6:16 p.m., in response to message #7 by Dave Shaffer (Arizona) Hi Dave, What you state matches what I remember from the old days, too. But it doesn't seem to be very popular, does it? So maybe I miss an important reason for single sided testing here, being so simple nobody thinks its worth explaining?

Re: Curve fitting question (OT) Message #9 Posted by Bart (UK) on 30 July 2009, 6:17 a.m., in response to message #1 by Walter B The Reduced Chi Square test is the Chi Square devided by the degrees of freedom. So it seems to me to be a "normalised" Chi Square, hence 1 being the optimum value. This makes it easier to present results without the reader having to worry about D.O.F., number of samples etc. The Chi Square test itself can be either a one sided (upper or lower) test, or a two sided test. However, only the upper one-sided test lends itself to the Reduced Chi Square. Bart

Re: Curve fitting question (OT) Message #10 Posted by Walter B on 30 July 2009, 5:07 p.m., in response to message #9 by Bart (UK) Hi Bart, I agree with everything you wrote but the last sentence. Nothing prevents me from dividing a lower Chi Square by its DF. Walter

Re: Curve fitting question (OT) Message #11 Posted by Bart (UK) on 30 July 2009, 6:24 p.m., in response to message #10 by Walter B Hi Walter The problem at the lower tail of the CS is that except for very large DOF, CS/DOF < 1, even if it is a bad fit. Bart

Re: Curve fitting question (OT) Message #12 Posted by Walter B on 30 July 2009, 6:51 p.m., in response to message #11 by Bart (UK) Hi Bart, CS/DOF will *always* stay <1 for the lower tail, as well as it will stay >1 for the upper tail d8) Both values will approach 1 for large DOF. Walter

Re: Curve fitting question (OT) Message #13 Posted by Bart (UK) on 30 July 2009, 8:04 p.m., in response to message #12 by Walter B Hi Walter, Sorry, I seem to have the wrong end of the "tail", RC2 is more suited for a lower tail GOF check, if <=1 is to be the acceptance criteria. However, as CS/DOF=1 is around the 60th percentile, the RC2 seems a very loose check for goodness of fit. As the 5th percentile is a more acceptable criteria, then RC2 < 0.78 for 100 DOF or <0.5 for 20 DOF. Let me add that I've used the Chi2 and looked into the RC2 when I saw your question this morning.

Re: Curve fitting question (OT) Message #14 Posted by Walter B on 1 Aug 2009, 5:21 p.m., in response to message #13 by Bart (UK) Just to illustrate what we are talking about: This is a diagram of Reduced Chi Square over Degrees Of Freedom. The lines are for different confidence levels and number of sides, and "einseitig" means single sided, while "zweiseitig" means (surprise!) double sided. Solid lines are for single sided, blue lines are for 99%, so broken lines are for double sided, and red lines are for 95%.

[ Return to Index | Top of Index ]

Go back to the main exhibit hall