The Under-specified problem

I think of the simple linear x y z problems of the previous pages as the attempt to find the single point in 3-space that satisfies the given equations. If the resultant square matrix is singular, then instead of finding a single point, we find a point on the line and we have an equation we can use to find the equation of the line, if desired, using (I - A‡A) z, or by directly examining the unused column(s) of the C matrix

Occasionally a matrix problem arises where there are more unknowns than equations. This happens, for instance, when combining Taylor's series for approximating the derivatives when using Finite-Differences (as I was wont to do back before learning Finite-Elements.) In this case, there's multiple possibilities but these are all related.

The equations

Here's a typical problem with one more unknown than equations:

The same vector notation applies, even if the A matrix is not square.

The matrix A is:

The vector x is:

The vector b is:

Inverting the non-square matrix

To solve for the x vector, we augment the matrix for row and column operations as before, but we must use Identity matrices of the appropriate rank:

Subtract row 1 from row 2.

Divide through row 2 by -4.

Subtract 2 of column 2 from column 3.

Subtract column 1 from column 2.

Add column 2 to column 3.

Divide through row 1 by 2.

The original matrix is completely reduced. What remains is an Identity matrix of rank 2, therefore we use 2 columns of the C matrix and 2 rows of the R matrix.

The Pseudo-inverse of the under-specified problem

Multiply the C matrix into the R matrix to obtain the A‡ pseudo-inverse.

Confirm the two generalized matrix properties

The first thing to notice is the A‡ pseudo-inverse is not square. We must be able to multiply this pseudo-inverse times our original A matrix on the right or left. For this to work, our pseudo-inverse must have as many columns as the original had rows, and as many rows as the original had columns.

Left and right hand inverse

Multiply our A‡ pseudo-inverse times the original A matrix.

Then multiply the original A matrix times this one to get A A‡ A which should equal A.

Therefore our A‡ pseudo-inverse meets the first property of the generalized matrices discussed earlier. It is the left-hand inverse of the original matrix.

Now check the pseudo-inverse and original for the second property - right-hand inverse. Multiply our intermediate matrix times the pseudo-inverse. This is A‡ A A‡ which should equal A‡.

Both properties are confirmed. This solution might not be the "least squares" solution obtained by Moore-Penrose, but the method doesn't suffer from the problems we saw earlier.

In the next section we look at applying this method to an even more under-specified problem. And here's a short note on how to go directly from this work to the parametric or symmetric equation of a line.