You learned to "undo" a matrix with its inverse G^{-1}: multiply by it and Gm = d unravels into m = G^{-1}d. Clean — but it only works when G is square and non-singular. Real inverse problems almost never oblige. A survey with 500 measurements and 3 unknowns gives a tall 500\times3 matrix; a blurry image reconstructed from too few measurements gives a wide one; and rank-deficient operators are everywhere. G^{-1} simply does not exist for any of them.

So do we give up on "inverting"? No — we generalize it. The Moore–Penrose pseudoinverse G^{+} extends the idea of an inverse to any matrix at all — tall, wide, square, full-rank or rank-deficient — and always returns one sensible, well-defined answer. Its defining promise:

Read that as a two-part guarantee. First, it fits the data as well as possible (least squares). Second, among all the equally-good fits, it picks the smallest model (minimum norm). One symbol, one answer, every shape of problem.

Two regimes, one symbol

The single symbol G^{+} quietly switches behaviour depending on the shape of the problem — and in each regime it reduces to a formula you may already know.

• Over-determined, full column rank (tall G): too many equations, no exact solution, so G^{+} gives the least-squares fit — and its formula is exactly the normal-equations operator, G^{+} = (G^{\mathsf T}G)^{-1}G^{\mathsf T}.
• Under-determined, full row rank (wide G): too few equations, infinitely many exact fits, so G^{+} gives the minimum-norm one — G^{+} = G^{\mathsf T}(GG^{\mathsf T})^{-1}.

And when G is square and invertible, both formulas collapse back to the ordinary inverse: G^{+} = G^{-1}. So the pseudoinverse doesn't replace what you know — it contains it as the special case, and reaches out to cover all the cases the ordinary inverse could not.

When G is rank-deficient, neither formula applies — the matrix you would need to invert (G^{\mathsf T}G or GG^{\mathsf T}) is itself singular. The pseudoinverse still exists, but to build it we need a decomposition that exposes the rank directly — the SVD, the universal route to G^{+}.

Worked example: over-determined — one unknown, two readings

Suppose one unknown m is measured twice, giving d = (2, 4)^{\mathsf T}, with model G = (1, 1)^{\mathsf T} (both readings should equal m). No single m gives both a 2 and a 4, so this is over-determined. Apply the tall formula: G^{\mathsf T}G = 2, G^{\mathsf T}d = 6, so

The pseudoinverse hands back the average of the two readings — the honest least-squares compromise between the conflicting measurements.

Worked example: under-determined — one reading, two unknowns

Now flip it. A single equation m_1 + m_2 = 6 constrains two unknowns: G = (1, 1) (a wide matrix), d = 6. There are infinitely many exact solutions — (6,0), (0,6), (-100, 106), all fit perfectly. Which one should an honest method report? The pseudoinverse picks the minimum-norm solution: the point on the solution line closest to the origin. Using the wide formula G^{+} = G^{\mathsf T}(GG^{\mathsf T})^{-1} with GG^{\mathsf T} = 2,

Out of infinitely many models that fit, it chooses (3, 3) — the least "extreme" one, splitting the total evenly and refusing to invent a wild swing between the two components that the single measurement never demanded. Minimum norm is the mathematical form of "don't claim more structure than the data forces on you."

What makes it the generalized inverse

You might wonder: couldn't you invent a dozen different "sort-of-inverses" for a non-square matrix? You can — but only one satisfies all four of the conditions Roger Penrose wrote down in 1955, and that uniqueness is what earns G^{+} the definite article. The conditions say, in plain terms:

• GG^{+}G = G — apply, undo, re-apply, and you are back where you started;
• G^{+}GG^{+} = G^{+} — the same for the pseudoinverse;
• GG^{+} and G^{+}G are both symmetric — they are honest orthogonal projections, not skewed ones.

Those last two projections are the geometric heart of the story. GG^{+} projects the data d orthogonally onto the column space (the reachable part) — that is the least-squares step. And G^{+}G projects any model onto the row space, throwing away the null-space component that the data can never see — that is the minimum-norm step. The two projections, packed into one matrix, are exactly why G^{+}d is simultaneously the best fit and the smallest model.

There is a single number that ties it back to the ordinary inverse. If G happens to be square with no zero singular values, both projections become the full identity, both Penrose "sandwich" identities collapse to GG^{+} = I, and G^{+} is forced to equal G^{-1}. The generalized inverse was the ordinary inverse all along, just stretched to survive the cases where the ordinary one falls apart.