newRPL - build 1255 released! [updated to 1299]

[Claudio L.]

(08-02-2019 07:57 PM)The Shadow Wrote:  Claudio, it might be worth looking at the PDQ algorithm as a replacement for the ->Q algorithm currently in use. Not only will it fix the "overshooting" problem I mentioned a while back, but it has other desirable properties. (Naturally for use in ->Q the tolerance would be taken from the system settings and one wouldn't bother calculating the error. But the full-blown PDQ might make a worthy command in its own right.)

I did replace the ->Q algorithm with PDQ (not published yet). Works nice but what should be the tolerance for ->Q? I did the obvious: Use the 10^(-prec) where prec is the current precision. However, the algorithm seems to have some glitches. Works great in some cases, bad in others for example:
12345678 13 / ->Q results in '12345678/13' (no surprises there)
12345678 7 / ->Q results in '1.763 667 051E31/9.999 993E24' (note: not all digits survived copy/paste from newRPL Desktop)
Doing ->NUM on this bizarre fraction, I get the original argument *exactly* with all 32 digits matching, so this fraction IS a correct result, just not what I thought I would get: '12345678/7'

(ohhhhhh.... moment) I just figured it out while typing this! Thanks to the forum...
If the argument has already 7 integer digits there's only 32-7=25 digits after the decimal point, so a tolerance of 10^-32 is simply "locking to zero" 7 digits after the last known digit.
So I'm adding 7 zeros after the number, and forcing the system to approximate *that* number.
Clearly, I just need to determine how many "fractional" digits are in the actual argument and use that as tolerance