WP-34S galUS-> l Precision

[Claudio L.]

(09-09-2016 08:05 AM)Paul Dale Wrote:  How does newRPL deal with unit conversions unrepresentable with decimal arithmetic?

Mustn't there be a precision loss involved with these using floating point arithmetic, even though they might be exact some of the time using rational arithmetic?



Pauli

Yes, there's a loss but operations are deferred to the last minute. For example, 1/7937 is stored as a "fake" unit that is defined as 7937, then if a unit needs a conversion of 1/7937, I just used that fake unit with a -1 exponent.
For newRPL this was an absolute must because of the variable precision in the system. If you set the system to 2000 digits, you expect 1/7937 to be represented with 2000 digits, so instead of trying to store them, they are computed at run time (you pay the price in speed, but you get the precision that's selected).