HP Forum Archive 10

[ Return to Index | Top of Index ]

calc precision Message #1 Posted by mapet on 10 Feb 2003, 9:51 a.m. Which of the HP calculators uses the highest precision?

Re: calc precision Message #2 Posted by Exp-PPC member on 10 Feb 2003, 10:03 a.m., in response to message #1 by mapet HP30S (24 digits), except for square root (12 digits)

Re: calc precision Message #3 Posted by Bill Wiese on 10 Feb 2003, 2:15 p.m., in response to message #2 by Exp-PPC member >>HP30S (24 digits), except for square >>root (12 digits) Wow. Why is sqrt(x) lagging? Common square root algorithms work very similarly to division & reciprocal algorithms. Unless they totally ran out of ROM space and picked a reduction in sqrt(X) precision as the best compromise, this looks like an oversight. I believe - it's been awhile - that there's a CORDIC-like shift/add algorithm for square root too - each iteration would deliver one digit (either binary bit, hex digit or BCD digit, according to base of numeration system and algorithm flavora) of result. Square root is fairly well-behaved too - unlike asymptotic tan(x) near 90deg and log(x), ln(x) near 0. Bill Wiese San Jose, CA

HP 9G? (nt) Message #4 Posted by mapet on 11 Feb 2003, 7:58 a.m., in response to message #2 by Exp-PPC member nt

Re: calc precision Message #5 Posted by hugh on 13 Feb 2003, 7:50 p.m., in response to message #2 by Exp-PPC member i have also noticed square root is less accurate. if you root with logs you can get more accuracy that the built in square root. that said, its not necessarily a recommendation for the hp30s.

[ Return to Index | Top of Index ]

Go back to the main exhibit hall