TI stat summations and guard digits Message #4 Posted by Karl Schneider on 30 Dec 2007, 4:13 p.m., in response to message #3 by Palmer O. Hanson, Jr.
Hi again, Palmer --
Happy holidays, and we all hope that your wife's surgery and recovery turns out for the best.
Quote:
My recollection is that the TI-55II displays eight digits but carries eleven digits internally.
Yes, I'm sure that you're correct, and those three extra guard digits contain the small inaccuracy from recalculation of the mean. However, it's possible for those calculations to be exact if the data are entered in ascending or descending order: {4, 5, 6, 7, 8} or {8, 7, 6, 5, 4}. The only intermediate mean average of up to five input data that is not exactly representable using two or fewer decimal points, is the mean of three data having an odd sum.
On a TI-55II or similar, try entering {4, 6, 7, 8, 5} using stat summation, or manually calculate
(((4 + 6 + 7)/3)*3 + 8 + 5)/5 - 6.
I get -1E-09 on my LED TI-30, which also has an eight-digit mantissa and three guard digits.
Reducing the possibility of overflow with an eight-digit mantissa was probably the reason for the TI-55II's statistical-summation methods. The guard digits were not to be considered reliable; on the LED TI-30, they certainly weren't:
http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv016.cgi?read=107358#107358
-- KS
Edited: 31 Dec 2007, 2:33 a.m.
|