FORTRAN floating point accuracy problems

04112016, 12:35 PM
Post: #41




RE: FORTRAN floating point accuracy problems
(04112016 12:03 PM)Dieter Wrote: In this case Excel indeed returns 20 for INT(100*dms). Indeed it does.. but it's an example of Excel 'fiddling'  it adjusted the results to match its decimal audience ;) Free42 Binary returns 20 only with 1e16  because then 5e16 + 1 equals 5e16. Excel really should use decimal arithmetic.. Cheers, Werner 

04122016, 06:48 PM
Post: #42




RE: FORTRAN floating point accuracy problems
(04112016 12:35 PM)Werner Wrote:(04112016 12:03 PM)Dieter Wrote: In this case Excel indeed returns 20 for INT(100*dms). Well, Excel and mathematics... that's two different worlds. Especially as far as more advanced functions are concerned. I remember how older versions handled the Normal quantile function – the results were inaccurate (about 5 digits) and below a certain limit the result simply was... ±500000. *facepalm* (04112016 12:35 PM)Werner Wrote: Free42 Binary returns 20 only with 1e16  because then 5e16 + 1 equals 5e16. Hmm.... Free42 Binary uses standard IEEE754 double precision and the computer's FPU, that's 16 valid digits at best. Now 1E15/(5E15+1) = 0,19999 99999 99999 960000..., which, even when rounded to 16 digits, equals 0,2. If IP(100*this) is 19, this of course is mathematically correct. But does Free42 Binary display 100x the above fraction as 19,999... or 20? So does it display 20 and at the same time return 19 as the integer portion of this? Things like these are the reason why I prefer Free42 Decimal. ;) Dieter 

04132016, 06:23 AM
Post: #43




RE: FORTRAN floating point accuracy problems
1E15/(5E15 +1) on Free42 Bin equals (hex digits
1.9999999999998 * 2^(3) In the display of course, this shows as 0.2, but after 1e8 x FP you see 9.99999996275e1 And the '6' in its decimal representation is really the 16th digit, not the 17th (the leading '1' is the implied bit, not a full digit), it doesn't get rounded. When you enter 0.2, it's hex representation is 1.999999999999A * 2^(3) Cheers, Werner 

04142016, 08:18 AM
Post: #44




RE: FORTRAN floating point accuracy problems
I don't think binary arithmetic is to blame here  rather, it's the lack of a true conversion to integer. Either truncation or rounding would be fine, so long as they are accurate to all digits. The problem here is a little extra "cosmetic" rounding which seems to be difficult or impossible to avoid.
It might be OK if you take the integer part and then check by subtraction that it's the right value  now you have a program instead of a formula. 

04192016, 06:39 AM
Post: #45




RE: FORTRAN floating point accuracy problems
Guys I have been preparing for a big holiday coming up so I haven't been able to get back to this thread. I am still interested and will be able to update in a few weeks. Sorry for the delay and thanks for all your comments and suggestions.
It ain't OVER 'till it's 2 PICK 

05032016, 11:54 AM
Post: #46




RE: FORTRAN floating point accuracy problems
Ok guys, I am almost back. Dieter, do you have an idea how to proceed now that we have your rounding function available?
Anybody else, ideas I should try? I'll go over the thread again since I have forgotten where we left off. Thanks. It ain't OVER 'till it's 2 PICK 

05032016, 12:47 PM
Post: #47




RE: FORTRAN floating point accuracy problems
(05032016 11:54 AM)HP67 Wrote: Ok guys, I am almost back. Dieter, do you have an idea how to proceed now that we have your rounding function available? Using the rounding function you could try something like this: Code: Function dms2dd(dms) Here "safedigits" is the number of significant decimals we can consider exact. That's 15 for IEEE754 DP and 16 in your case. The algorithm essentially determines how many digits after the decimal point can be assumed exact (15 resp. 16 minus the number of digits of the integer part) and rounds the mm.ss part accordingly. Dieter 

05032016, 01:02 PM
(This post was last modified: 05032016 01:04 PM by HP67.)
Post: #48




RE: FORTRAN floating point accuracy problems
Thanks, I'll try this and post back when I can.
It would be nice if there were some way of determining safedigits environmentally, without having to code a constant, so the code would be somewhat portable. It ain't OVER 'till it's 2 PICK 

05032016, 02:56 PM
Post: #49




RE: FORTRAN floating point accuracy problems
(05032016 01:02 PM)HP67 Wrote: It would be nice if there were some way of determining safedigits environmentally, without having to code a constant, so the code would be somewhat portable. Some languages feature a function that returns the smalles possible machine epsilon. Others could use something like this: Code: Function safedigits() However, things may be slightly different with base16 encoded numbers like the ones used by your hardware: cf. here. Dieter 

05042016, 11:05 AM
Post: #50




RE: FORTRAN floating point accuracy problems
Code:
Outstanding! Thanks, Dieter! I'll get back to you on how I made out with your program that calculates the epsilon later. It ain't OVER 'till it's 2 PICK 

05042016, 01:34 PM
(This post was last modified: 05042016 02:05 PM by Dieter.)
Post: #51




RE: FORTRAN floating point accuracy problems
(05042016 11:05 AM)HP67 Wrote: Outstanding! Thanks, Dieter! Fine. There is just one issue: if the input is very small, i.e. dms is a fraction of a second, or maybe even something below 1E–16, the input may be rounded down to zero. This can be avoided by replacing the respective lines by this: Code: If dms >= 1 Then If there was a FLOOR function the test was not required at all: k=FLOOR(log10(dms). Maybe you can write one in Assembler. ;) Now add a few lines that handle dms<0 (there is a DSIGN function in Fortran IV) or dms=0 (simply return zero), and you're done. Edit: this could be a solution. Code: Function dms2dd(dms) If I get it right, dsign transfers the sign of the second argument to the first one, i.e. dsgn(x, y) = abs(x) * sign(y), which is quite handy here. Dieter 

05042016, 02:50 PM
(This post was last modified: 05042016 02:52 PM by HP67.)
Post: #52




RE: FORTRAN floating point accuracy problems
(05042016 01:34 PM)Dieter Wrote: Fine. There is just one issue: if the input is very small, i.e. dms is a fraction of a second, or maybe even something below 1E–16, the input may be rounded down to zero. This can be avoided by replacing the respective lines by this: Actually I had a FLOOR function in FORTRAN that was based on the less than optimal IDINT function that I replaced with my handydandy assembler DINT. So this is available now. This is what I have been using, does it look ok or can it be improved: Code:
(05042016 01:34 PM)Dieter Wrote: Now add a few lines that handle dms<0 (there is a DSIGN function in Fortran IV) or dms=0 (simply return zero), and you're done. Given our beautiful results above is this necessary? If you think it's a good idea I'll look into that later since I have shut the machine off already. Thanks again. It ain't OVER 'till it's 2 PICK 

05042016, 06:04 PM
Post: #53




RE: FORTRAN floating point accuracy problems
(05042016 02:50 PM)HP67 Wrote: Actually I had a FLOOR function in FORTRAN that was based on the less than optimal IDINT function that I replaced with my handydandy assembler DINT. So this is available now. This is what I have been using, does it look ok or can it be improved: It can be improved so that it returns correct results for integer x<0. The given implementation will return FLOOR(–3) = –4, which of course is wrong. You might test if x<0 and x≠int(x). (05042016 02:50 PM)HP67 Wrote: Given our beautiful results above is this necessary? The suggested changes provide correct results for arguments that are extremely small, zero or negative. I think this makes sense. Dieter 

06042016, 06:42 PM
Post: #54




RE: FORTRAN floating point accuracy problems
Thanks. I have no idea how I could have missed that. Something looked wrong to me. Fixed!
I'll get back to the other improvements asap but have been unable to get back to this for a long time since some big things have been going on. It ain't OVER 'till it's 2 PICK 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)