MPINVERT: Moore-Penrose Inverse of a Matrix

[Han]

The SVD decomposition of a matrix can also be used to compute the pseudo-inverse of a matrix. If
\[ A = U \cdot \Sigma \cdot V^T \]
where \( \sigma_i \) are the singular values of the diagonal matrix \( \Sigma \), and \( U \) and \( V \) are unitary matrices.

The pseudo-inverse of \( A \) is
\[ A^{+} = V \cdot \Sigma^{+} \cdot U^T \]
where \( \Sigma^{+} \) is obtained by replacing the diagonal entries \( \sigma_i \) with \( 1/\sigma_i \) for \( \sigma_i > \epsilon \) and 0 otherwise and transposing. If \( A \) is non-singular, then all \( \sigma_i \) are non-zero and the pseudo-inverse is the same as the regular inverse. On the other hand, if \( A \) is singular, then only the "non-zero" (greater than \( \epsilon \)) singular values are inverted in computing \( A^{+} \).

It's a bit more powerful in that it even handles cases when \( A^TA \) or \( AA^T \) are not invertible. The SVD has lots of neat applications! What are they? Well, that's a google exercise left to the diligent reader :-)