Unexpected result calculating the determinant of a singular matrix (42S)

[Thomas Okken]

(10-21-2019 08:41 AM)Moggul Wrote:  The Free 42 on my phone gives 0. Different methodology or different precision?

Free42 uses LU decomposition to calculate determinants as well. The implementation is based on the one from "Numerical Recipes in C," without the "TINY" fudge factor when encountering a zero pivot. If the algorithm were identical to the one in the HP-42S, I'd expect the same kinds of errors, just smaller because of the extra precision, but in actual fact, it returns exactly zero, just like the 48G.

(10-21-2019 04:23 AM)Valentin Albillo Wrote:  

(10-21-2019 03:11 AM)Thomas Okken Wrote:  The 48G gets exactly zero on [[-.2 .1 .3][.1 .2 .1][.3 .1 -.2]] as well. Wouldn't that defeat the cheat?

No. It checks that the values of all elements are integer, not their types. The floating-point constant 2. has the integer value 2.

I think you misread my post. I tried Dave's example, divided by 10. Not inexact numbers, but actual non-integer values.