HP Prime numerical precision in CAS and HOME Message #1 Posted by Javier Goizueta on 12 Nov 2013, 1:16 p.m.
One funny thing I've noticed about the Prime: the HOME and CAS worlds differ in its internal numerical precision (when CAS operates in approximate mode):
In Home the 12decimal digit precision of former HP models is maintained:
1(4/31)*3 => .00000000001
But in the CAS, 47 internal bits of precision is used:
1.0(4./3.1.)*3. => 1.42108547152E14
Which is 2^46.
I'd had expected 53 bits (for IEEE 64bit floating point). Anybody can offer more insight into this?
Edited: 12 Nov 2013, 1:17 p.m.
