HP Prime: Rounding error in determinant Message #1 Posted by Stephan Matthys on 25 Oct 2013, 8:49 a.m.
Let's assume the following matrix:
[[4 5 7] [8 9 0] [5 6 2]]
The determinant (as given by my Prime (hardware & emulator, both running version 2013 8 13. Rev:5016)) is 13.
The transposed matrix is:
[[4 8 5] [5 9 6] [7 0 2]]
The corresponding determinant is 13.0000000002
Why? Shouldn't they be the same?
btw: my HP15C also makes a small error (2e8) in the determinant of the transposed matrix...
