Post Reply 
FORTRAN floating point accuracy problems
05-04-2016, 11:05 AM
Post: #50
RE: FORTRAN floating point accuracy problems
Code:

T1DMS2DD

 TEST          DMS       EXPECT       RESULT         DIFF

    1   89.1115000   89.1875000   89.1875000    0.0000000
    2   12.1500000   12.2500000   12.2500000    0.0
    3   33.3000000   33.5000000   33.5000000    0.0
    4   71.0030000   71.0083333   71.0083333    0.0000000
    5   42.2453000   42.4147222   42.4147222    0.0000000
    6   38.4225000   38.7069444   38.7069444    0.0000000
    7   29.3030000   29.5083333   29.5083333    0.0000000
    8    0.4949000    0.8302777    0.8302778    0.0000001
    9   75.1500000   75.2500000   75.2500000    0.0
   10   43.2230000   43.3750000   43.3750000    0.0000000
   11    9.3345000    9.5625000    9.5625000    0.0000000
   12   33.5752200   33.9645000   33.9645000    0.0
   13   13.0724420   13.1234500   13.1234500    0.0000000
   14   21.3000000   21.5000000   21.5000000    0.0
   15   59.4721120   59.7892000   59.7892000    0.0
   16   65.1100960   65.1836000   65.1836000    0.0000000

Outstanding! Thanks, Dieter!

I'll get back to you on how I made out with your program that calculates the epsilon later.

It ain't OVER 'till it's 2 PICK
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: FORTRAN floating point accuracy problems - HP67 - 05-04-2016 11:05 AM



User(s) browsing this thread: 1 Guest(s)