The Over-specified problem

Sampled data is one place where you may have many more values (the b vector and equations) than unknowns (the x vector). In this case, the Moore-Penrose solution gives a relatively small matrix to invert because the transpose of A times A will be a square matrix with rows and columns the same as the number of columns of A. (Sampled data in the current application on which I'm working typically has over 2000 rows of sampled data and 8 unknowns.) Inverting an 8x8 matrix is certainly quicker than inverting one that is 2000x8. Unfortunately, even sampled data processed this way can give a matrix that is singular.

When there are more equations than unknowns (the opposite of that discussed on the previous page), there should be no variability in the solution. If there is, then some of the equations (rows of sampled data) are simply combinations of other rows.

The equations

Here's a typical problem with one more equation than unknowns:

The same vector notation applies, even if the A matrix is not square.

The matrix A is:

The vector x is:

The vector b is:

Inverting the non-square matrix

To solve for the x vector, we augment the matrix for row and column operations as before:

Move row 3 up to row 1.

Add column 1 to column 2.

Subtract 2 of row 1 from row 2.

Add row 1 to row 3.

Subtract row 3 from row 2.

Subtract 2 of row 2 from row 3.

Creating the A‡ matrix

The original matrix is reduced to an identity of rank 2, so we multiply the 2 columns of the C matrix times the 2 rows of the R matrix to produce the A‡ pseudo-inverse matrix.

To check that this is the inverse, multiply this matrix times the original:

Other valid solutions

The Identity means there is no variability in the solution. Over-specified problems are this way. But is this the only solution?

A different sequence of column and row operations produced this inverse:

Multiply this matrix times the original:

Therefore, once again, the inverse is not unique. If all that is needed is a solution, this method is sufficient. If the least-squares solution is needed, use the Moore-Penrose method but use this for inverting the product (AT A), because even sampled data may yield a matrix that is singular.

The next page has some JavaScript that allows you to enter a matrix of your choosing, and invert it using this method. This was previously written in Java that was run on a Servlet, but I couldn't get a static IP where I'm living now, and don't want to pay to have a Servlet hosted in a server somewhere that costs me $. I'm cheap.

There is also a page that hosts the links to the various sources I still have in the languages I've used in the past. I consider this code completely open source with no warranty whatsoever.