FORTRAN floating point accuracy problems

[Dieter]

(04-11-2016 12:35 PM)Werner Wrote:  

(04-11-2016 12:03 PM)Dieter Wrote:  In this case Excel indeed returns 20 for INT(100*dms).

Indeed it does.. but it's an example of Excel 'fiddling' - it adjusted the results to match its decimal audience ;-)
(...)
Excel really should use decimal arithmetic..

Well, Excel and mathematics... that's two different worlds. Especially as far as more advanced functions are concerned. I remember how older versions handled the Normal quantile function – the results were inaccurate (about 5 digits) and below a certain limit the result simply was... ±500000. *facepalm*

(04-11-2016 12:35 PM)Werner Wrote:  Free42 Binary returns 20 only with 1e16 - because then 5e16 + 1 equals 5e16.

Hmm....
Free42 Binary uses standard IEEE754 double precision and the computer's FPU, that's 16 valid digits at best.
Now 1E15/(5E15+1) = 0,19999 99999 99999 960000..., which, even when rounded to 16 digits, equals 0,2.
If IP(100*this) is 19, this of course is mathematically correct. But does Free42 Binary display 100x the above fraction as 19,999... or 20? So does it display 20 and at the same time return 19 as the integer portion of this?

Things like these are the reason why I prefer Free42 Decimal. ;-)

Dieter