Question for Parisse about CAS floating point

[jgoizueta]

(04-06-2015 01:44 PM)compsystems Wrote:  hp-prime calculator

approx(1519/99) => 15.3434343434 10 digits =(

The 15.3434343434 result of the Prime uses 10 decimal places but has 12 digits of precision (counting the integer part).

Actually, the 48 internal bits of the approximate CAS numbers can hold 15 decimal digits (and preserve their value when converted back to decimal)

The actual CAS value 1519.0/99.0 could be presented as 15.3434343434343 with 15 digit precision, but the Prime refuses to do so. (you can check the actual internal value with Joe Horn's hex program and see that all those digits are there)

I don't care about the precision of the approximate numbers in the CAS (it is larger than the 12 digits of HOME). What worries me is that each result is truncated (after being computed with the accuracy of a IEEE double), so numeric computations in the CAS may not be as well behaved as in home...

Edited to mention the hex program

Edit: oh, now I realize that the CAS values are shown to 12 decimal digits for consistency with the HOME world.