FORTRAN floating point accuracy problems

[HP67]

(05-04-2016 01:34 PM)Dieter Wrote:  Fine. There is just one issue: if the input is very small, i.e. dms is a fraction of a second, or maybe even something below 1E–16, the input may be rounded down to zero. This can be avoided by replacing the respective lines by this:

Code:

If dms >= 1 Then
   k = int(log10(dms))
Else
   k = int(log10(dms)) - 1
End If

If there was a FLOOR function the test was not required at all: k=FLOOR(log10(dms).
Maybe you can write one in Assembler. ;-)

Actually I had a FLOOR function in FORTRAN that was based on the less than optimal IDINT function that I replaced with my handy-dandy assembler DINT. So this is available now. This is what I have been using, does it look ok or can it be improved:

Code:


      DOUBLE PRECISION FUNCTION DFLOOR(X)                               00040003
      REAL*8 X                                                          00050000
      REAL*8 DINT                                                       00070004
C                                                                       00080004
      DFLOOR = DINT(X)                                                  00090004
      IF (X .LT. 0.0D0) DFLOOR = DFLOOR - 1.0D0                         00100002
      RETURN                                                            00110000
      END                                                               00120000

(05-04-2016 01:34 PM)Dieter Wrote:  Now add a few lines that handle dms<0 (there is a DSIGN function in Fortran IV) or dms=0 (simply return zero), and you're done.

Edit: this could be a solution.

Code:

Function dms2dd(dms)
Const safedigits = 16
adms = Abs(dms)

If adms >= 1 Then
   k = Int(log10(adms))
Else
   If adms = 0 Then
      k = 0
   Else
      k = Int(log10(adms)) - 1
   End If
End If

n = safedigits - 3 - k

dd = Int(adms)
mmss = Round(100 * (adms - dd), n)
mm = Int(mmss)
ss = (mmss - mm) * 100

dms2dd = dsign(dd + mm / 60 + ss / 3600, dms)

End Function

If I get it right, dsign transfers the sign of the second argument to the first one, i.e. dsgn(x, y) = abs(x) * sign(y), which is quite handy here.

Dieter

Given our beautiful results above is this necessary? If you think it's a good idea I'll look into that later since I have shut the machine off already.

Thanks again.