The Museum of HP Calculators

HP Forum Archive 20

[ Return to Index | Top of Index ]

RM mode and its effects on RRM determinant of AM1 matrix
Message #1 Posted by Gene wright on 22 Aug 2011, 5:55 p.m.

RM Mode       Determinant
-------   ---------------------
  0       0.99999 711935 38228
  1       0.99999 720675 30552
  2       0.99999 709507 09280
  3       1.00000 009178 43880
  4       0.99999 711925 74617
  5       1.00000 895891 46320
  6       0.99999 718613 18536

Looks like RM 3 is closest to 1. Fairly significantly closer, it appears.

      
Re: RM mode and its effects on RRM determinant of AM1 matrix
Message #2 Posted by Paul Dale on 22 Aug 2011, 6:16 p.m.,
in response to message #1 by Gene wright

RM 3 is round half up. It being closer is most likely a fluke.

This does show that the algorithm isn't overly sensitive to rounding mode which is goodness.

- Pauli

[ Return to Index | Top of Index ]

Go back to the main exhibit hall