The Museum of HP Calculators

HP Forum Archive 17

 HP 35s Standard DeviationMessage #1 Posted by John B. Smitherman on 18 Sept 2007, 10:49 p.m. I wonder if someone would mind checking the standard deviation calculation on a 35s to see if HP has fixed the algorithm. Enter 1,000,000 , 1,000,001 and 1,000,002 and find the sample standard deviation. The correct answer should be 1 but as has been discussed before the 33s provided an error message. Thanks, John

 Re: HP 35s Standard DeviationMessage #2 Posted by bt_schmidt on 18 Sept 2007, 10:58 p.m.,in response to message #1 by John B. Smitherman If I'm doing it right, I get: STAT ERROR ...bt

 Re: HP 35s Standard DeviationMessage #3 Posted by John B. Smitherman on 18 Sept 2007, 11:43 p.m.,in response to message #2 by bt_schmidt Thanks bt. From my testing it looks like this is done correctly in the 49G+/50G and 17bii+. As discussed previously this problem only shows up when trying to calculate small differences between large numbers. Regards, John

 Re: HP 35s Standard DeviationMessage #4 Posted by designnut on 19 Sept 2007, 12:30 a.m.,in response to message #3 by John B. Smitherman I read in some Hp instructions to subtract the large value before doing statistics. It involves the square of the input and clearly the calculator does not have that accuracy. Sam

 Re: HP 35s Standard DeviationMessage #5 Posted by Walter B on 19 Sept 2007, 1:37 a.m.,in response to message #3 by John B. Smitherman Please see this previous thread. Besides this, there may be more in earlier discussions. HTH

 Re: HP 35s Standard DeviationMessage #6 Posted by Hal Bitton in Boise on 19 Sept 2007, 1:43 p.m.,in response to message #1 by John B. Smitherman 48GX gets it right (returns 1) as well. Regards, Hal

 Re: HP 35s Standard DeviationMessage #7 Posted by Thomas Radtke on 19 Sept 2007, 1:54 p.m.,in response to message #6 by Hal Bitton in Boise The 19B does it right, too. Maybe because these models do the statistics based on a list?

 Re: HP 35s Standard DeviationMessage #8 Posted by bill platt on 19 Sept 2007, 3:03 p.m.,in response to message #1 by John B. Smitherman This is beyond the range of the machine. It doesn't work on a 32sii nor on a 15c either. It is not a flaw.

 Re: HP 35s Standard DeviationMessage #9 Posted by Karl Schneider on 21 Sept 2007, 2:40 a.m.,in response to message #8 by bill platt John Smitherman posted, Quote: Enter 1,000,000 , 1,000,001 and 1,000,002 and find the sample standard deviation. The correct answer should be 1 but as has been discussed before the 33s provided an error message. Bill Platt replied, Quote: This is beyond the range of the machine. It doesn't work on a 32sii nor on a 15c either. It is not a flaw. Bill is right; the "STAT ERROR" or "Error 2" message is attributable to roundoff errors in the statistical summation. What's ironic is that the failure to produce a result was made possible by extended-precision calculation, albeit using standard-precision inputs. Here are the tabulations: ```Xi (Xi)^2 (exact) (Xi)^2 (result retaining 10 or 12 significant digits) 1,000,000 1,000,000,000,000 1,000,000,000,000 1,000,001 1,000,002,000,001 1,000,002,000,000 1,000,002 1,000,004,000,004 1,000,004,000,000 Summations: S(X) S(X^2) (exact) S(X^2) (using 10 or 12-digit results) 3,000,003 3,000,006,000,005 3,000,006,000,000 ([S(X)]^2) (exact result obtained using 13- or 15-digit extended precision) 9,000,018,000,009 ([S(X)]^2)/3 (exact result obtained using 13- or 15-digit extended precision) 3,000,006,000,003 ``` The exact calculation sqrt ([S(X^2) - ([S(X)]^2)/n] / [n-1]) of sample standard deviation is sqrt [(3,000,006,000,005 - 3,000,006,000,003) / (3-1)] = 1 The "erroneous" calculation is based on 13- or 15-digit internal computaion that utilizes the S(X^2) summations that were rounded off after each datum entry: sqrt [(3,000,006,000,000 - 3,000,006,000,003) / (3-1)] = ERROR (The HP-15C's extended-precision calculations for statistical summations are utilized within an example in the HP-15C Advanced Functions Handbook listed on pages 208-211.) If extended precision had not been employed for the problem discussed here, both terms would have been rounded to 3,000,006,000,000 -- yielding the incorrect result of zero, instead of an error. -- KS Edited: 26 Sept 2007, 2:09 a.m. after one or more responses were posted

 Re: HP 35s Standard DeviationMessage #10 Posted by Palmer O. Hanson, Jr. on 22 Sept 2007, 8:38 p.m.,in response to message #9 by Karl Schneider You wrote: "Use of extended-precision calculations for standard deviation is described in the HP-15C Advanced Functions Handbook on page 208." Page 208 of my August 1982 copy only says "Program B exploits a tricky property of the Sum- and Sum+ keys whereby certain calculations can be carried out to 13 significant digits before being rounded back to 10." Is there a later revision which gives specific details for the calculatioin of standard deviation?

 "Tricky" properties for HP-15C Standard DeviationMessage #11 Posted by Karl Schneider on 23 Sept 2007, 12:47 a.m.,in response to message #10 by Palmer O. Hanson, Jr. Hi, Palmer -- No, my version of the HP-15C AFH gives no further details. I realized after posting that "described" was not a very good term, but didn't edit before you responded. I have edited the post now. I was also well aware that these methods were the basis of your "Cadillac Quadratic Solver for the hp 33s" article from last year: -- KS

 Re: "Tricky" properties for HP-15C Standard DeviationMessage #12 Posted by Palmer O. Hanson, Jr. on 23 Sept 2007, 9:31 p.m.,in response to message #11 by Karl Schneider Quote: No, my version of the HP-15C AFH gives no further details. I am sorry about that. I was hoping for something more definitive on the "tricky properties" than what I had been able to figure out by trial and error. Many years ago TI was very close-mouthed about the implementation of the statistcs routines in the TI-59 until the user community figured out how to download the instruction set. Then they published a discussion of the routines. The routines did include an HIR 20 command which was some sort of branching command which could not be made to work in user memory -- or at least, no one ever figured out how to do it. The TI discussion included mention of the HIR 20 command, but their explanation was about as informative as a Milton Friedman dissertation on the state of the economy.

 Re: "Tricky" properties for HP-15C Standard DeviationMessage #13 Posted by Karl Schneider on 26 Sept 2007, 2:43 a.m.,in response to message #12 by Palmer O. Hanson, Jr. Quote: I was hoping for something more definitive on the "tricky properties" than what I had been able to figure out by trial and error. Without scrutinizing the program on the HP-15C AFH, pp. 208-211, I do notice a few things: Only Sigma- is used; each time it is, the contents of register R7 (which contain the XY summation) are recalled. The quadratic equation is defined on page 205 as c - 2bz + az2, making the discriminant d = b2 - ac The basic purpose appears to be the calculation of "b2-ac" and similar terms to 13-digit extended precision before being rounded back down to 10 digits. This procedure increases accuracy when the terms in the subtraction are nearly equal. It's kind of related to the residual matrix calculation (B - AC). -- KS Edited: 4 Oct 2007, 2:33 p.m.

 Re: HP 35s Standard DeviationMessage #14 Posted by John B. Smitherman on 23 Sept 2007, 3:51 p.m.,in response to message #9 by Karl Schneider Thanks Karl. As usual your post was very thorough and insightful. Perfect benchmarks are not easy to find. I believe that users want HP and other calculators manufacturers to provide accurate and robust algorithms that make the best use of the hardware selected. It's interesting how tests keep showing up from various users that highlight where some machines fall short of our expectations, especially near or around the limits of the machine. Regards, John

 Re: HP 35s Standard DeviationMessage #15 Posted by Bill (Smithville, NJ) on 19 Sept 2007, 3:21 p.m.,in response to message #1 by John B. Smitherman Hi John, I don't have a 35S, so can't directly comment on it, but the following is a quote from page 237 of the HP-42S manual that addresses this: ```Limitations on Data Values The calculator might be unable to perform some statistical calculations if your data values differ by a relatively small amount. To avoid this, you should normalize your data by entering values as the difference from one value (such as the mean). This difference must then be added back to any calculations of the mean. For example, if your x-values were 776999, 777000, and 777001, you should enter the data as -1, 0, and 1; then add 777000 back to the relevant results. ``` Does the HP-35S manual include any paragrah such as this? Bill Edited: 19 Sept 2007, 3:22 p.m.

 Re: HP 35s Standard DeviationMessage #16 Posted by Gene Wright on 19 Sept 2007, 3:37 p.m.,in response to message #15 by Bill (Smithville, NJ) Yes, on page 12-10. Gene

 Re: HP 35s Standard DeviationMessage #17 Posted by Antonio Maschio (Italy) on 19 Sept 2007, 4:04 p.m.,in response to message #15 by Bill (Smithville, NJ) The same on the HP-15C manual. -- Antonio

 Re: HP 35s Standard DeviationMessage #18 Posted by Namir on 19 Sept 2007, 3:47 p.m.,in response to message #1 by John B. Smitherman Most calculators will fail this kind of calculations, unless you shift the values into a more appropriate range. Namir

 Re: HP 35s Standard DeviationMessage #19 Posted by John B. Smitherman on 20 Sept 2007, 7:53 a.m.,in response to message #18 by Namir Most will fail but not all. My HP 30s with s/n CN0351 yields the correct answer. Regards, John

 Re: HP 35s Standard DeviationMessage #20 Posted by Palmer O. Hanson, Jr. on 22 Sept 2007, 8:44 p.m.,in response to message #19 by John B. Smitherman Quote: Most will fail but not all. My HP 30s with s/n CN0351 yields the correct answer. Actually, it is many HP calculators which will fail because they carry only ten digits. Calculators from other companies such as the Casio fx-7000G, the TI-59, TI-66, TI-68, TI-95 and Durabrand 828 which carry 13 digits will not fail this particular test. They will, of course, fail tests which require more than thireen digits. I wonder, was this test proposed by a TI Guy?

 Re: HP 35s Standard DeviationMessage #21 Posted by John B. Smitherman on 23 Sept 2007, 3:35 p.m.,in response to message #20 by Palmer O. Hanson, Jr. Thanks Palmer. Through brute force I found that most fail testing 10,000,000 , 10,000,001 and 10,000,002 while the HP 50G and 17Bii+ fail at 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002. Regards, John

 17Bii+ SDEV testMessage #22 Posted by Karl Schneider on 24 Sept 2007, 2:48 p.m.,in response to message #21 by John B. Smitherman Quote: ... while the HP 50G and 17Bii+ fail (to calculate sample standard deviation of the data) 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002. Hi again, John -- The last two of these data require 13 digits, which cannot be represented as non-integer values on these calculators. The calculation will fail even if summation methods are not used. Does this mean that the HP-17Bii+ accepts excess digits, as the RPL-based models have done, and now the HP-35s does? If so, more ammunition for my case... -- KS

 Re: 17Bii+ SDEV testMessage #23 Posted by John B. Smitherman on 24 Sept 2007, 3:41 p.m.,in response to message #22 by Karl Schneider Hi Karl. In your note you asked: Quote: Does this mean that the HP-17Bii+ accepts excess digits, as the RPL-based models have done, and now the HP-35s does? If so, more ammunition for my case... To further support your case when I enter 1,000,000,000,002 the 49G+ and 17bii+ display 1E12. I then subtract 2 and they both display 999,999,999,998. So, the excess digits are being dropped. Regards, John

 Re: HP 35s Standard DeviationMessage #24 Posted by Palmer O. Hanson, Jr. on 25 Sept 2007, 12:09 a.m.,in response to message #21 by John B. Smitherman Quote: the HP 50G and 17Bii+ fail at 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002. My TI-83+ yields the correct results for that but fails at 1E13, 1E13 + 1 and 1E-13 + 2 . I am fairly sure that it does not calculate SDEV using the summations but rather uses the sum of the squares of the differences from the mean. My TI-86 yields the correct results for 1E13, 1E13 + 1 and 1E13 + 2 but fails at 1E14, 1E14 + 1 and 1E14 + 2. Edited: 25 Sept 2007, 12:50 a.m.

 Re: HP 35s Standard DeviationMessage #25 Posted by Bruce Bergman on 19 Sept 2007, 4:04 p.m.,in response to message #1 by John B. Smitherman Free42 correctly calculates it. ;-) Also, the 17bii and HP-50g calculate it correctly. thanks, bruce Edited: 19 Sept 2007, 4:09 p.m.

 Re: HP 35s Standard DeviationMessage #26 Posted by Norris on 19 Sept 2007, 6:43 p.m.,in response to message #25 by Bruce Bergman Like many HP calculator issues, the different methods used to determine standard deviation have been discussed here before. This long thread started out as a discussion of the 33S polar conversion bug, but digressed into a discussion of standard deviation.

 Re: HP 35s Standard DeviationMessage #27 Posted by Palmer O. Hanson, Jr. on 20 Sept 2007, 2:51 a.m.,in response to message #26 by Norris The HP-41 yields the message "OUT OF RANGE" for the given problem. Page 101 of the HP-41C Owner's Handbook and Programming Guide includes the statement "Note: If your data {xi} or {yi} contains many redundant leading digits, you should refrain from copying them into the calculator. For example, if your x-data is {999999999, 1000000001, 1000000002}, you should enter the x-data as {-1, 1, 2} and add the redundant digits to any x-related answer produced."

 Re: HP 35s Standard DeviationMessage #28 Posted by Martin Pinckney on 21 Sept 2007, 10:50 a.m.,in response to message #1 by John B. Smitherman This exercise gave me new respect for my 38G, which gave the correct result.

Go back to the main exhibit hall