HP-71B internal summation weakness in Math ROM
|
06-05-2020, 07:43 PM
(This post was last modified: 06-05-2020 07:45 PM by J-F Garnier.)
Post: #17
|
|||
|
|||
RE: HP-71B internal summation weakness in Math ROM
(06-05-2020 05:43 PM)ttw Wrote: I need to correct my previous statement. In fact, there is no rounding mechanism that can take a result rounded to 15 bits then correctly round that to 12 bits. The expected error may be better, but there are always cases where fewer than 12 bits will be correct. Yes, it's the problem of the double rounding. An example with complex numbers on a 28S or 42S: (2,4.995E-12)*(1,1) returns (2, 2.00000000001) also with the new HP71 Math 2b, but NOT on the original Math 1A that returns (2,2) as do the HP-32S/SII. I think HP considered that these marginal cases were less important than the (visible) incorrect rounding such as in the DOT example I gave above. J-F (edited: it's easy to find examples when you know where to look at :-) |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)