[HP12C] Angular coefficent from linear regression off by 1

05062016, 01:11 PM
Post: #19




RE: [HP12C] Angular coefficent from linear regression off by 1
(05062016 07:50 AM)Csaba Tizedes Wrote: [ 6step solution using r and s ] In this particular case it's good enough of course, as this is the exact 10digit result. ;) The method you suggested is short and elegant. However, it uses the regression coefficient and the two standard deviations, where all three are internally calculated using square roots. That's why this method is prone to slight errors in the last digit, just as the method suggested in the manual. And that's why the suggested direct solution using the six registers is inherently more exact. As long as these data do not have too many significant digits (which usually is the case with empirical data) the results are exact. Even in the last digit. ;) So you can choose between a shorter and a potentially more exact solution. Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)