Post Reply 
FORTRAN floating point accuracy problems
03-31-2016, 02:45 PM (This post was last modified: 03-31-2016 02:55 PM by HP67.)
Post: #20
RE: FORTRAN floating point accuracy problems
Thanks very much.

Mostly I'm using this to convert decimal time back and forth. .001 seconds should be plenty of resolution. I'll be happy with a reliable hh.mmsst for now.

I think a rounding function and a double precision truncation function would be more generally useful but I'll study what you came up with. I can anticipate a few problems but fortunately I am almost always wrong about math so maybe this time it will work in my favor.

FORTRAN IV doesn't have any integer division except for integer types and doesn't have 8 byte integers. When we get back to dividing again won't we go through the same or similar conversions/promotions as before? So I am expecting problems. Again, I'm probably wrong.

I'll play around with your suggestion. Eventually I would like to come up with an 8 byte truncation function for reals. The rounding function is very handy also.

In understanding your declarations, Excel VBA long means 8 byte int? No comparable type in FORTRAN IV. Will a 4 byte int work? I think you explained that it will, so I don't understand why you chose long in your code.

It ain't OVER 'till it's 2 PICK
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: FORTRAN floating point accuracy problems - HP67 - 03-31-2016 02:45 PM



User(s) browsing this thread: 1 Guest(s)