Gaussian Elimination
Imagine a system with not two unknowns but twenty, or two thousand — the kind of thing an
engineer solves when balancing forces in a bridge, or a bank solves when pricing a portfolio of
loans. Guessing is hopeless. What you want is a recipe: a sequence of mechanical
steps that, followed exactly, always lands on the answer — no cleverness, no luck, no dead ends.
That recipe is Gaussian elimination. Use
row operations
to carve the augmented matrix into a staircase shape (row-echelon form) — zeros
marching down and to the left below a diagonal of leading entries — and then read the unknowns off
from the bottom row upward, a process called back-substitution.
It always works, it never guesses, and it scales to a system of any size. That is precisely why
it is not just a classroom exercise: it is the algorithm quietly running, right now, inside
spreadsheet "solver" buttons, structural-engineering software, and every scientific computing
library on Earth, chewing through millions of equations a second without anyone watching.
Notice the shape of the plan: first make the problem easier (the staircase), then make it
trivial (back-substitution). That two-phase split — grind forward, then read backward —
is a pattern you'll meet again and again once you start writing algorithms of your own. Elimination
is arguably the oldest example of it in mathematics.
The three legal moves — and why they're safe
Elimination is built entirely out of the three
row operations.
The reason they're allowed at all is that each one is reversible: whatever it
does can be exactly undone by another legal move. A step that can always be undone can never
smuggle in a new solution or erase a real one — so the solution set at the very end is guaranteed
to be identical to the solution set you started with.
- Swap two rows — reordering equations obviously can't change what satisfies
them.
- Scale a row by a non-zero number — multiplying both sides of one
equation by the same non-zero constant keeps it true for exactly the same values.
- Combine — add a multiple of one row to another. If two equations both hold,
so does their sum (or any weighted mix of them); and because the move is reversible, you can
always subtract the same multiple back off.
Any finite sequence of these three moves turns a system into an equivalent one — same unknowns, same solution set, friendlier shape.
Elimination just applies "combine" (and occasionally "swap") again and again, in a very
deliberate order: use each pivot row to blast zeros into every entry beneath it, moving left to
right, one column at a time, until the whole matrix is a staircase.
Here's the "combine" move proved safe in miniature. Suppose some pair
(x, y) satisfies both 2x + y = 5 and
x - y = 1. Add -2 times the second equation
to the first: the left side becomes (2x + y) - 2(x - y) = 3y, and the
right side becomes 5 - 2(1) = 3, so 3y = 3.
Any (x, y) solving the original two equations automatically solves this
new one too — nothing was invented. And because you could just as easily add
+2 times the second equation back to undo it, no solution was thrown
away either. That two-way safety, repeated across every step, is the entire reason the final
staircase has exactly the solutions the messy starting system had.
Worked example: three equations, three unknowns
Here is the full recipe run end to end on
2x + y - z = 8, \qquad -3x - y + 2z = -11, \qquad -2x + y + 2z = -3.
Write it as an augmented matrix and go hunting for zeros, column by column.
\left[\begin{array}{ccc|c} 2 & 1 & -1 & 8 \\ -3 & -1 & 2 & -11 \\ -2 & 1 & 2 & -3 \end{array}\right]
Step 1 — clear column 1 below the pivot. The pivot is the top-left
2. Replace R_2 with
2R_2 + 3R_1, and replace R_3 with
R_3 + R_1. Both are legal "combine" moves — each row is just replaced
by a mix of itself and the pivot row.
\left[\begin{array}{ccc|c} 2 & 1 & -1 & 8 \\ 0 & 1 & 1 & 2 \\ 0 & 2 & 1 & 5 \end{array}\right]
Step 2 — clear column 2 below the new pivot. The pivot is now the
1 in row 2. Replace R_3 with
R_3 - 2R_2:
\left[\begin{array}{ccc|c} 2 & 1 & -1 & 8 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & -1 & 1 \end{array}\right]
That's row-echelon form — a genuine staircase, zeros below every pivot. Now
back-substitute, bottom row first: row 3 reads -z = 1,
so z = -1. Feed that into row 2:
y + (-1) = 2 \Rightarrow y = 3. Feed both into row 1:
2x + 3 - (-1) = 8 \Rightarrow x = 2. Done —
x=2,\ y=3,\ z=-1 — checked by plugging straight back into all three
original equations.
Step it through: a smaller system, one move at a time
The three-equation example above moves fast on the page, so slow it right down on a
two-equation system: 2x + y = 5, x - y = 1.
Click through each row operation below and watch the numbers change — by the final frame the
matrix reads x = 2, y = 1 directly, no
algebra left to do.
Worked example: when elimination reveals no solution
Elimination doesn't just find answers — run honestly, it also reveals when there
isn't one, without you ever having to guess. Try
x + y = 3, \qquad 2x + 2y = 11.
Scale the first row by 2 so its x and
y coefficients match the second row, then combine — replace
R_2 with R_2 - 2R_1:
\left[\begin{array}{cc|c} 1 & 1 & 3 \\ 2 & 2 & 11 \end{array}\right]
\;\longrightarrow\;
\left[\begin{array}{cc|c} 1 & 1 & 3 \\ 0 & 0 & 5 \end{array}\right]
The bottom row now reads 0x + 0y = 5, i.e.
0 = 5 — an outright contradiction, since no values of
x and y can make zero equal five. There is
no solution. Geometrically the two equations describe two parallel lines
(y = -x + 3 and y = -x + 5.5) that never
meet — but you didn't need to draw a single line to find that out. The algorithm told you, in the
arithmetic itself, the moment a row collapsed to nonsense.
Reading the staircase
Keep going past plain row-echelon form — clear the entries above each pivot too, and
scale every pivot down to a lone 1 — and you reach the
reduced row-echelon form: an
identity
block sitting beside the answer column. That is the goal whenever a unique solution exists,
because at that point the matrix is quite literally saying "x = \ldots,
y = \ldots, z = \ldots" — nothing left to
solve.
Three outcomes, three shapes: a full staircase all the way to an identity block means one unique
solution. A row that collapses all the way to 0 = 0 (true no matter
what) means that equation added no new information — infinitely many solutions remain. A row that
collapses to something impossible, like the 0 = 5 above, means no
solution exists at all. The shape of the staircase tells you everything — which is exactly the
idea behind
rank.
Only the three operations above are allowed — swap, scale by a non-zero number, or add a
multiple of one row to another. Two mistakes sneak in disguised as "obviously fine":
-
Multiplying a row by zero is not scaling — it's destruction. "Scale" requires a
non-zero constant precisely because multiplying by zero throws the whole equation away
and replaces it with the useless 0 = 0, silently losing a real
constraint. It is not reversible — you can never scale a zero row back to what it was — which is
the tell-tale sign a move is illegal.
-
Losing track of which row is the current pivot. A very common slip when working
by hand: after using row 2 as the pivot to clear column 2, a tired eliminator sometimes reuses
the original row 2 again lower down, instead of the already-updated one. Elimination
only stays correct if every operation acts on the most recently updated version of each
row — always work top to bottom, and always re-read the current matrix before each step, never
the matrix from two steps ago.
Not first, no — though he does deserve the name for a different reason.
Carl Friedrich Gauss streamlined and popularised
the method in the early 1800s while crunching astronomical orbit calculations, and it stuck to
him in the West. But essentially the same staircase-and-back-substitute procedure appears in
the Nine Chapters on the Mathematical Art, a Chinese mathematical text compiled roughly
two thousand years earlier, complete with instructions for arranging counting
rods in a rectangular array and combining rows — the same algorithm, the same idea, discovered
independently on the other side of the world.
The method's second life is just as remarkable: it is the workhorse buried inside almost every
piece of numerical software that exists — spreadsheet "Solver" add-ins, structural and circuit
simulators, machine-learning libraries fitting a line to data. Modern versions add tricks like
choosing the largest available pivot first to fight rounding error, but the bones of the
algorithm — turn it into a staircase, then read upward — are exactly what you just did by hand.
That rounding-error trick is called partial pivoting, and it matters more than it
sounds. A computer only stores numbers to a limited number of digits, so dividing by a pivot that
happens to be tiny can wildly amplify a rounding error into a hugely wrong answer. Real solvers
scan a column for its largest available entry and swap it to the top before using it as a
pivot — the same "swap" move you already know, deployed defensively. A bridge's engineers trust the
software precisely because it is doing careful, disciplined elimination, not because it is doing
anything more mysterious than what you just worked through by hand.