HP Prime Miscalculating
10-24-2015, 12:48 PM
Post: #12
 retoa Member Posts: 168 Joined: Jan 2015
RE: HP Prime Miscalculating
(10-23-2015 05:41 AM)Joe Horn Wrote:  Long answer: It's because Home uses BCD (which can represent 3.6 and 3.24 exactly) and CAS uses binary floating point (which can't). When you type 3.6 in Home, it's EXACTLY 3.6, but when you type 3.6 in CAS, it actually generates this value:

3.599999999999994315658113919198513031005859375 (exactly)

... because that's the largest number less than or equal to 3.6 which is representable with a 48-bit mantissa. For what it's worth, it's stored internally in hex scientific notation as 1.CCCCCCCCCCCCp+1 where "p" means "times 2 to the power of".

So the prime uses a 48 bits mantissa to work with binary floating point, and not 52 like in IEEE 754 double precision. How will be a number represented?
I mean in double precision you have 1 bit for the sign, 11 for the exponent and 52 for the mantissa. What about the Prime? If it works with 64 bits and the mantissa is truncated from 52 to 48 what are the 4 "missing" bits used for?

And how is a BCD number represented? I would say 48 bits for the mantissa as we have 12 digits (which is the same as in CAS, so it seems consistent). And the other bits?