FORTRAN floating point accuracy problems

[Dieter]

(03-30-2016 01:10 PM)HP67 Wrote:  Do you have any idea what's going on? Why should rounding to more places cause it to round down instead of up?

Yes, I think I know what's going on here: Your Fortran seems to support a precision of about ten decimal digits. That's why rounding to 13 digits does not help. I think your DROUND function is fine, just replace the 4 digit to 8 digit rounding. Or, if you are sure that nothing beyond 1/100 second occurs, simply set it to 6 digit rounding.

You may test this easily: simply write a short program that prints a periodic value like 1/3 or 1/9 with 15 digits. For instance, simply let the program print 1.0D0/3.0D0 with 15 decimal places and see what you get. If it's something like 0,33333 33334 8855... you know that 31 bits or 9-10 decimal places is all you got. ;-)

(03-30-2016 01:10 PM)HP67 Wrote:  If I specify rounding to 4 places the results improve a lot but are still worse than when unrounded:

Sure. Rounding to 4 places will work fine if 0.mmss does not have more than 4 digits, i.e. no fractions of a second. And indeed for these cases your results are fine, only the "EXPECT" values are wrong again: dms2dd(71,0030) = 71,0083333.... and dms2dd(0,4949) = 0,83027777...., or dms2dd(65,11010) = 65,18361111... which is exactly what your program returns. The "expected" values 71,0080, 0,83020 and 65.18600 are WRONG!

Actually all cases with not more than 4 decimals are perfectly correct. Even those with 5 places (i.e. fractions of a second) are exact if the argument is rounded to full seconds (4 digits). For instance, dms2dd(13,07244) actually evaluates dms2dd(13,0724), so mm.ss = 7,2400000 and the result for this is 13,12333. Adjust the roundoff function to 5 places (or 6, or 8) and these will return exact results as well.

Now please try rounding to 6 or 8 digits and report what you got.
And, more important, check your results with a reliable reference, e.g. an HP. ;-)

Dieter