A quick precision test

[walter b]

(06-08-2014 12:44 PM)pito Wrote:  Take 1.0000001 and do x^2 27times..

UPDATE:
Basic precision:

Code:

Calculator        Display           Display-INT
=================================================
W.Alfa (Ref)      674530.470741     .4707410845
-------------------------------------------------
Canon X Mk I Pro  674530.4707       .4707399243
WP-34s            674530.47054      .4705396874
WP-31s            674530.47054      .4705396874

WP 34S and WP 31S share the same SW in that matter so it's no wonder they both return the same single precision result. After DBLON, however, the WP 34S displays 674530.470741 with a fractional part (FP) of 0.470741084559 And looking at its full precision via < and >, we see it is 0.4707410845593826891847277722). Anybody selling more, FWIW?

d:-)

P.S.: Wolfram alpha returns 674 530.470 741 084 559 382 689 178... at it's machine precision. If I counted correctly, that's a deviation of 10[super]-26[/super], FWIW.