HP Forums - Weird result with Matrix determinant

I wanted to calculate the determinant of the matrix
$$\begin{pmatrix} 1 & 0 & 2 & 8 \\ -2 & 1 & 1 & 5 \\ 6 & 3 & 5 & -4 \\ 10 & 4 & 3 & 0 \end{pmatrix}$$
Using Standard number format,
Prime Home says -647,000000002
Prime CAS says -647

Home is pretty close but what kind of algorithm can produce such a funny result to calculate a determinant with only integers?

Other question is: do we have an explanation for Floating and Rounded Number Formats? Those are recent additions not documented in the user guide.

Hi everyone,

When i enter the same matrix on my 28S DET gives me -647,000000002 too !
The 48GX, 49G, 50G give the good answer -647 !

Regards,

Damien

(03-17-2015 08:16 PM)Tugdual Wrote: [ -> ]Using Standard number format,
Prime Home says -647,000000002
Prime CAS says -647

Home is pretty close but what kind of algorithm can produce such a funny result to calculate a determinant with only integers?

Well, like any other device with limited working precision (12 resp. 15 digits internally) your calculator will cause a more or less significant roundoff error with every single operation. Evaluating the determinant of a matrix with numerical methods, i.e. not by Cramer's rule or similar exact solutions that are not suitable for larger matrices, usually is done via the well-known Gauss algorithm resp. a LU-decomposition, leaving the product of the diagonal elements as the desired result.

All this involves dozens, if not hundreds of multiplications, divisions and additions. Try a simple example like 2/3 – 2*(1/3) and you'll see how limited the accuracy of even a close-to-perfect calculator can be. A tiny error in the least significant digit is a very good result here. With other matrices the error can grow much, much larger.

Dieter

Thank you for the explanation Dieter. When using CAS I indeed see fractions on the diagonal of the U matrix so this definitely explains the variation.