03-27-2016, 03:02 PM

I'm getting some unacceptable results from rounding in old FORTRAN. I know there are some very talented mathematicians on the board here so I wonder if you guys would be willing to help me. To say I'm bad at math would be an understatement! I have no idea where things are going wrong.

I'm trying to accomplish something very simple in a language which has very little in the way of helpful functions to do what I want. Given a floating point input value for degrees, minutes, seconds in the form dd.mmsst... I want to produce the value in decimal degrees. This is no problem of math even for me. The problem is extracting the digits with only a truncation function and nothing else helpful in the repetoire as far as I can see.

Here is the piece of code:

and here are the results of some tests:

You can see in test case #3 above there is a big problem extracting the minutes. The code winds up with 29 instead of 30 because of the truncation function.

I find this kind of problem is coming up a lot in various calculations I'm doing and I don't know how to code around it.

Can anybody please explain how I can rearrange the computations or do something differently to avoid this problem?

Thanks.

I'm trying to accomplish something very simple in a language which has very little in the way of helpful functions to do what I want. Given a floating point input value for degrees, minutes, seconds in the form dd.mmsst... I want to produce the value in decimal degrees. This is no problem of math even for me. The problem is extracting the digits with only a truncation function and nothing else helpful in the repetoire as far as I can see.

Here is the piece of code:

Code:

DOUBLE PRECISION FUNCTION DMS2DD (DMS)

REAL*8 DMS, DD, MM, SS

SS = (DMS - IDINT(DMS)) * 100.0D0

SS = ((SS - IDINT(SS)) * 100.0D0) / 60.0D0

MM = (IDINT((DMS - IDINT(DMS)) * 100.0D0) + SS) / 60.0D0

DMS2DD = IDINT(DMS) + MM

RETURN

END

and here are the results of some tests:

Code:

TEST DMS EXPECTED RESULT DIFFERENCE

1 89.11150 89.18750 89.18750 0.00000

2 12.15000 12.25000 12.25000 0.00000

3 33.30000 33.50000 33.51111 0.01111

4 71.00300 71.00830 71.00833 0.00003

5 42.24530 42.41470 42.41472 0.00002

6 38.42250 38.70690 38.70694 0.00004

7 29.30300 29.50830 29.50833 0.00003

8 0.49490 0.83030 0.83028 0.00002

9 75.15000 75.25000 75.25000 0.00000

10 43.22300 43.37500 43.37500 0.00000

11 9.33450 9.56250 9.56250 0.00000

12 33.57522 33.96450 33.96450 0.00000

13 13.07244 13.12345 13.12345 0.00000

14 21.30000 21.50000 21.50000 0.00000

15 59.47211 59.78920 59.78920 0.00000

16 65.11010 65.18360 65.18360 0.00000

You can see in test case #3 above there is a big problem extracting the minutes. The code winds up with 29 instead of 30 because of the truncation function.

I find this kind of problem is coming up a lot in various calculations I'm doing and I don't know how to code around it.

Can anybody please explain how I can rearrange the computations or do something differently to avoid this problem?

Thanks.