(12C) Decimal to Fraction

[Albert Chan]

(08-07-2018 07:29 AM)Dieter Wrote:  Even with 12 digit precision the error of 577/408 produced by the program is the same as for 816/577. The former even has a lower denominator. In FIX 5 mode the result 577/408 is not returned only because the fifth decimal is rounded up (1,41422)

I made a mistake with 816/577. It is not a continued fraction of sqrt(2)

With this mistake, I discovered my misconception with continued fraction.
I thought "best" fraction with limited denominator is always a continued fraction ... FALSE

577/408 over - estimated sqrt(2): 2.1239014e-6
816/577 under-estimated sqrt(2): 2.1238982e-6 --> lowered error by by 0.00015%

Your Pi example illustrate even better, fraction with denominator <= 100:
continued fraction of pi => 22/7 = 3.142857
pi.limit_denominator => 311/99 = 3.141414 --> about 7X more accurate