[HP12C] Angular coefficent from linear regression off by 1

01192019, 05:36 PM
Post: #30




RE: [HP12C] Angular coefficent from linear regression off by 1
(05052016 07:07 AM)Csaba Tizedes Wrote: 01: 0 This regression slope code is neat ! (05062016 01:11 PM)Dieter Wrote: The method you suggested is short and elegant. However, it uses the regression coefficient and the two standard deviations, where all three are internally calculated using square roots. That's why this method is prone to slight errors in the last digit, just as the method suggested in the manual. Manual suggested method, slope = f(1)  f(0), may lose *many* significant digits, due to subtraction cancellation. 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)