Weird result with Matrix determinant - Printable Version +- HP Forums (https://www.hpmuseum.org/forum) +-- Forum: HP Calculators (and very old HP Computers) (/forum-3.html) +--- Forum: HP Prime (/forum-5.html) +--- Thread: Weird result with Matrix determinant (/thread-3409.html) Weird result with Matrix determinant - Tugdual - 03-17-2015 08:16 PM I wanted to calculate the determinant of the matrix $$\begin{pmatrix} 1 & 0 & 2 & 8 \\ -2 & 1 & 1 & 5 \\ 6 & 3 & 5 & -4 \\ 10 & 4 & 3 & 0 \end{pmatrix}$$ Using Standard number format, Prime Home says -647,000000002 Prime CAS says -647 Home is pretty close but what kind of algorithm can produce such a funny result to calculate a determinant with only integers? Other question is: do we have an explanation for Floating and Rounded Number Formats? Those are recent additions not documented in the user guide. RE: Weird result with Matrix determinant - Damien - 03-17-2015 10:09 PM Hi everyone, When i enter the same matrix on my 28S DET gives me -647,000000002 too ! The 48GX, 49G, 50G give the good answer -647 ! Regards, Damien RE: Weird result with Matrix determinant - Dieter - 03-17-2015 11:29 PM (03-17-2015 08:16 PM)Tugdual Wrote:  Using Standard number format, Prime Home says -647,000000002 Prime CAS says -647 Home is pretty close but what kind of algorithm can produce such a funny result to calculate a determinant with only integers? Well, like any other device with limited working precision (12 resp. 15 digits internally) your calculator will cause a more or less significant roundoff error with every single operation. Evaluating the determinant of a matrix with numerical methods, i.e. not by Cramer's rule or similar exact solutions that are not suitable for larger matrices, usually is done via the well-known Gauss algorithm resp. a LU-decomposition, leaving the product of the diagonal elements as the desired result. All this involves dozens, if not hundreds of multiplications, divisions and additions. Try a simple example like 2/3 – 2*(1/3) and you'll see how limited the accuracy of even a close-to-perfect calculator can be. A tiny error in the least significant digit is a very good result here. With other matrices the error can grow much, much larger. Dieter RE: Weird result with Matrix determinant - Tugdual - 03-18-2015 08:47 PM Thank you for the explanation Dieter. When using CAS I indeed see fractions on the diagonal of the U matrix so this definitely explains the variation.