Still having problems with specific input values. I tried my own version before I looked at yours and I tried yours also. Unsurprisingly yours worked a little better. But we are still having problems with specific values. BTW the output is shown to 7 decimal places but the input and expected values are only to 6 values. I showed extra digits since last time it seemed important.
Aside from specific problematic values (.15, .3) this works better than required. So, in that sense much progress. Thanks.
Code:
LEVEL 21.8 ( JUN 74 ) OS/360 FORTRAN H DATE 16.094/15.37.38
COMPILER OPTIONS - NAME= MAIN,OPT=02,LINECNT=50,SIZE=0000K,
SOURCE,EBCDIC,LIST,DECK,NOLOAD,MAP,EDIT,NOID,XREF
ISN 0002 DOUBLE PRECISION FUNCTION DMS2DD (DMS) 00010013
ISN 0003 REAL*8 DMS, DDD 00020013
ISN 0004 INTEGER MMSS, MM, SS 00030013
C 00040013
ISN 0005 DDD = IDINT(DMS) 00050013
ISN 0006 MMSS = (DMS - DDD) * 1000000000 00060013
ISN 0007 MM = MMSS / 10000000 00070013
ISN 0008 SS = MOD(MMSS, 10000000) 00080013
C 00090013
ISN 0009 DMS2DD = DDD + MM / 60.0D0 + SS / 360000000.0D0 00100013
ISN 0010 RETURN 00110013
ISN 0011 END 00120013
T1DMS2DD
TEST DMS EXPECT RESULT DIFF
1 89.1115000 89.1875000 89.1875000 0.0000000
2 12.1500000 12.2500000 12.2611111 0.0111111
3 33.3000000 33.5000000 33.5111111 0.0111111
4 71.0030000 71.0083330 71.0083333 0.0000003
5 42.2453000 42.4147220 42.4147222 0.0000002
6 38.4225000 38.7069440 38.7069444 0.0000004
7 29.3030000 29.5083330 29.5083333 0.0000003
8 0.4949000 0.8302770 0.8302778 0.0000008
9 75.1500000 75.2500000 75.2611111 0.0111111
10 43.2230000 43.3750000 43.3750000 0.0000000
11 9.3345000 9.5625000 9.5625000 0.0000000
12 33.5752200 33.9645000 33.9645000 0.0000000
13 13.0724420 13.1234500 13.1234500 0.0000000
14 21.3000000 21.5000000 21.5111111 0.0111111
15 59.4721120 59.7892000 59.7892000 0.0000000
16 65.1100960 65.1836000 65.1836000 0.0000000
******************************** BOTTOM OF DATA ************************************************************************************
LEVEL 21.8 ( JUN 74 ) OS/360 FORTRAN H DATE 16.094/15.38.43
COMPILER OPTIONS - NAME= MAIN,OPT=02,LINECNT=50,SIZE=0000K,
SOURCE,EBCDIC,LIST,DECK,NOLOAD,MAP,EDIT,NOID,XREF
ISN 0002 DOUBLE PRECISION FUNCTION DMS2DD (DMS) 00060013
ISN 0003 REAL*8 DMS 00070013
C 00080013
ISN 0004 INTEGER IDMS, DD, MM, SS 00090013
C 00100013
ISN 0005 IDMS = DMS * 1.0D7 00110013
C 00120013
ISN 0006 DD = IDMS / 1.0D7 00130013
ISN 0007 MM = (IDMS / 1.0D5) - (DD * 100) 00140013
ISN 0008 SS = IDMS - (DD * 1.0D7) - (MM * 1.0D5) 00150013
C 00160013
ISN 0009 DMS2DD = DD + MM / 60.0D0 + SS / 36.0D5 00170013
ISN 0010 RETURN 00190013
ISN 0011 END 00200013
T1DMS2DD
TEST DMS EXPECT RESULT DIFF
1 89.1115000 89.1875000 89.1874997 0.0000003
2 12.1500000 12.2500000 12.2611108 0.0111108
3 33.3000000 33.5000000 33.5111108 0.0111108
4 71.0030000 71.0083330 71.0083331 0.0000001
5 42.2453000 42.4147220 42.4147219 0.0000001
6 38.4225000 38.7069440 38.7069442 0.0000002
7 29.3030000 29.5083330 29.5083331 0.0000001
8 0.4949000 0.8302770 0.8302775 0.0000005
9 75.1500000 75.2500000 75.2611108 0.0111108
10 43.2230000 43.3750000 43.3749997 0.0000003
11 9.3345000 9.5625000 9.5624997 0.0000003
12 33.5752200 33.9645000 33.9644997 0.0000003
13 13.0724420 13.1234500 13.1234497 0.0000003
14 21.3000000 21.5000000 21.5111108 0.0111108
15 59.4721120 59.7892000 59.7891997 0.0000003
16 65.1100960 65.1836000 65.1835997 0.0000003
******************************** BOTTOM OF DATA ************************************************************************************
This floating point hardware implementation uses a base 16 internal representation. Do you have any idea how we might change the algorithm to use powers of 16 instead of powers of 10 or if that has any chance of working better?
I want to figure out how to code a double precision (8 byte) truncation function as I mentioned earlier. Your rounding routine is really great. If my implementation wasn't hampered by the existing 4 byte truncation function maybe all this would work a lot better?
I have figured out how to convert the internal representation to decimal but there is some loss of precision on my HP calcs since they only carry 12 significant digits. I have not figured out enough about the math behind this to understand what I need to do to truncate the decimal portion while it's still in the internal base 16 format. I'm sure anybody mathematically inclined could see it right away but I'm still banging my head.
Update: I see an instruction that could help with the the 8 byte truncation function. Will update on that in the next few days hopefully.
Search
Member List
Calendar
Help
/
FORTRAN floating point accuracy problems
