wp34s - FIB problem (DBLON)

[Gerson W. Barbosa]

(11-24-2018 08:37 PM)Paul Dale Wrote:  

(11-23-2018 10:45 AM)Gerson W. Barbosa Wrote:  So the only problem is it is not accurate enough in double precision mode for certain applications. Well, not really a problem once we are aware of that.

We deliberately made no guarantees about accuracy in double precision. That the FIB function is in XROM and working at double precision pretty much guarantees it will be off in the last place or so. Flash was full so I had to pull the C implementation that worked in higher precision.

I agree ROUNDI is better for your purposes.

The accuracy of the FIB function in double precision is indeed very good. By "not accurate enough" I meant it failed giving the exact integer results I expected for integer arguments. I am sorry I haven't made myself clear about that.

Gerson.