Post Reply 
HP Prime Miscalculating
10-24-2015, 12:48 PM
Post: #12
RE: HP Prime Miscalculating
(10-23-2015 05:41 AM)Joe Horn Wrote:  Long answer: It's because Home uses BCD (which can represent 3.6 and 3.24 exactly) and CAS uses binary floating point (which can't). When you type 3.6 in Home, it's EXACTLY 3.6, but when you type 3.6 in CAS, it actually generates this value:

3.599999999999994315658113919198513031005859375 (exactly)

... because that's the largest number less than or equal to 3.6 which is representable with a 48-bit mantissa. For what it's worth, it's stored internally in hex scientific notation as 1.CCCCCCCCCCCCp+1 where "p" means "times 2 to the power of".

So the prime uses a 48 bits mantissa to work with binary floating point, and not 52 like in IEEE 754 double precision. How will be a number represented?
I mean in double precision you have 1 bit for the sign, 11 for the exponent and 52 for the mantissa. What about the Prime? If it works with 64 bits and the mantissa is truncated from 52 to 48 what are the 4 "missing" bits used for?

And how is a BCD number represented? I would say 48 bits for the mantissa as we have 12 digits (which is the same as in CAS, so it seems consistent). And the other bits?

Thank you for the answer

Reto
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
HP Prime Miscalculating - luc4as - 10-23-2015, 02:35 AM
RE: HP Prime Miscalculating - Joe Horn - 10-23-2015, 05:41 AM
RE: HP Prime Miscalculating - retoa - 10-24-2015 12:48 PM
RE: HP Prime Miscalculating - parisse - 10-25-2015, 09:13 AM
RE: HP Prime Miscalculating - Tim Wessman - 10-23-2015, 02:54 PM
RE: HP Prime Miscalculating - retoa - 10-23-2015, 04:43 PM
RE: HP Prime Miscalculating - Tim Wessman - 10-23-2015, 04:54 PM
RE: HP Prime Miscalculating - Archanus - 10-23-2015, 07:38 PM
RE: HP Prime Miscalculating - Nigel (UK) - 10-23-2015, 10:12 PM
RE: HP Prime Miscalculating - Wes Loewer - 11-01-2015, 01:16 PM
RE: HP Prime Miscalculating - Vtile - 11-01-2015, 02:05 PM
RE: HP Prime Miscalculating - Wes Loewer - 11-04-2015, 05:57 PM
RE: HP Prime Miscalculating - Vtile - 11-04-2015, 08:36 PM
RE: HP Prime Miscalculating - CyberAngel - 09-02-2018, 04:32 PM
RE: HP Prime Miscalculating - retoa - 10-23-2015, 09:33 PM
RE: HP Prime Miscalculating - retoa - 10-23-2015, 10:04 PM
RE: HP Prime Miscalculating - Archanus - 10-24-2015, 02:20 AM
RE: HP Prime Miscalculating - parisse - 10-24-2015, 06:37 AM
RE: HP Prime Miscalculating - dan_h - 10-24-2015, 02:14 PM
RE: HP Prime Miscalculating - retoa - 10-30-2015, 01:34 PM
RE: HP Prime Miscalculating - parisse - 10-31-2015, 06:34 AM
RE: HP Prime Miscalculating - Vtile - 10-31-2015, 10:50 AM
RE: HP Prime Miscalculating - Joe Horn - 11-01-2015, 02:12 AM
RE: HP Prime Miscalculating - parisse - 11-06-2015, 07:55 AM



User(s) browsing this thread: 1 Guest(s)