WP 34S vs. DM42 decimal128 differences?

[Paul Dale]

(03-26-2018 09:51 PM)Nigel (UK) Wrote:  Is this a change worth making (once I've done some more extensive testing, and looked at the code for tan x as well)? Or is there a downside other than speed to using these extra digits?

I'd be concerned about overflowing the stack in other functions that call this. E.g. the gamma code can call this and gamma in turn is called from the statistical routines which are close to the limit already.

Using a smaller number of digits in single precision mode might also fall awry of this function's use elsewhere. If later digits carry an error they could cascade into the first sixteen.

I'll have to have a think about it...


Pauli