Hand someone a pile of vectors pointing every which way and ask them to describe the space those
vectors span, and things get messy fast — every direction leans on every other one. An
orthonormal basis — vectors that are mutually perpendicular and each of length
1 — is the nicest kind of basis to work with instead. Coordinates come
straight from dot
products, with no matrix to invert, no guessing.
The Gram–Schmidt process is the assembly line that turns the messy pile into the
tidy one. Given any basis, it produces an orthonormal basis spanning
exactly the same space, by a single repeated idea, applied one vector at a time: take the
next vector in line and strip away whatever part of it is already covered by the
directions you've built so far — project it onto each of them and subtract that
projection off. Whatever survives is genuinely new, and automatically perpendicular to everything
that came before.
The one move: subtract the projection
Think of it as an assembly line with one station per vector. The first vector rolls through
untouched — it becomes the first fixed direction. Every vector after that has to stop at
every previous station and have its "shadow" along that station's direction removed
before moving on. Whatever's left when it reaches the end of the line is guaranteed perpendicular
to every direction fixed so far — that shadow-removal step is exactly the
projection of one
vector onto another. Start with a basis
\mathbf{v}_1, \mathbf{v}_2, \dots:
- Keep the first direction: \mathbf{u}_1 = \mathbf{v}_1.
- For each next \mathbf{v}_k, subtract its projection onto
every direction already chosen:
\mathbf{u}_k = \mathbf{v}_k - \sum_{j < k} \operatorname{proj}_{\mathbf{u}_j}(\mathbf{v}_k), \qquad \operatorname{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\,\mathbf{u}.
What's left, \mathbf{u}_k, is perpendicular to all the earlier ones.
Finally normalise each to length 1:
\mathbf{e}_k = \mathbf{u}_k / \lVert \mathbf{u}_k \rVert, and you have an
orthonormal basis.
Here is the key step in the plane: \mathbf{v}_2 minus its shadow on
\mathbf{v}_1 leaves a piece exactly perpendicular to
\mathbf{v}_1. Step through it:
A worked example
Take \mathbf{v}_1 = (3, 1) and
\mathbf{v}_2 = (1, 2). Set
\mathbf{u}_1 = (3, 1). Then
\operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_2) = \frac{\mathbf{v}_2 \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\,\mathbf{u}_1 = \frac{3 + 2}{9 + 1}\,(3,1) = \tfrac{1}{2}(3,1) = (1.5,\, 0.5),
\mathbf{u}_2 = \mathbf{v}_2 - \operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_2) = (1,2) - (1.5, 0.5) = (-0.5,\, 1.5).
Check they are perpendicular: \mathbf{u}_1 \cdot \mathbf{u}_2 = (3)(-0.5) + (1)(1.5) = -1.5 + 1.5 = 0. ✓
Finish the job: normalise. Orthogonal isn't orthonormal yet — divide each vector
by its own length. Here
\lVert \mathbf{u}_1 \rVert = \sqrt{3^2+1^2} = \sqrt{10} and
\lVert \mathbf{u}_2 \rVert = \sqrt{(-0.5)^2+1.5^2} = \sqrt{2.5}, so
\mathbf{e}_1 = \frac{\mathbf{u}_1}{\sqrt{10}} \approx (0.949,\, 0.316), \qquad \mathbf{e}_2 = \frac{\mathbf{u}_2}{\sqrt{2.5}} \approx (-0.316,\, 0.949).
Both now have length 1, and
\mathbf{e}_1 \cdot \mathbf{e}_2 \approx 0.949(-0.316) + 0.316(0.949) = 0
(up to rounding) — a genuine orthonormal basis, built from a lopsided starting pair.
Why bother? Coordinates for free. This is the payoff promised at the start:
once a basis is orthonormal, finding any vector's coordinates is nothing but a couple of dot
products — no matrix to invert. Take \mathbf{w} = (4, 4) and read off
its coordinates in the new basis \{\mathbf{e}_1, \mathbf{e}_2\}:
c_1 = \mathbf{w}\cdot\mathbf{e}_1 \approx 4(0.949) + 4(0.316) \approx 5.06, \qquad c_2 = \mathbf{w}\cdot\mathbf{e}_2 \approx 4(-0.316) + 4(0.949) \approx 2.53.
Rebuilding c_1\mathbf{e}_1 + c_2\mathbf{e}_2 lands back on
(4, 4) (up to rounding) — exactly the point of straightening the
basis out in the first place.
Try Gram–Schmidt on \mathbf{v}_1 = (2, 4) and
\mathbf{v}_2 = (1, 2) — notice
\mathbf{v}_2 is just half of \mathbf{v}_1,
so really they only point along one shared line. Set
\mathbf{u}_1 = (2,4); then
\operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_2) = \frac{2+8}{4+16}\,(2,4) = \tfrac12(2,4) = (1,2), \qquad \mathbf{u}_2 = (1,2) - (1,2) = (0,0).
The leftover is exactly the zero vector — Gram–Schmidt is quietly telling you that
\mathbf{v}_1 and \mathbf{v}_2 never
spanned a plane at all, only a line. This is the "bonus diagnostic" from the warning box below:
the process can't manufacture a second perpendicular direction out of thin air if the inputs
didn't actually contain one.
The idea scales up without any new rules — each new vector just has more shadows to subtract.
Take \mathbf{v}_1 = (1,1,0),
\mathbf{v}_2 = (1,0,1), and
\mathbf{v}_3 = (0,1,1) in \mathbb{R}^3.
As before, \mathbf{u}_1 = (1,1,0) and subtracting
\mathbf{v}_2's shadow on \mathbf{u}_1
gives \mathbf{u}_2 = (0.5,\, -0.5,\, 1). For
\mathbf{u}_3, subtract \mathbf{v}_3's
shadow on both earlier vectors:
\mathbf{u}_3 = \mathbf{v}_3 - \operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_3) - \operatorname{proj}_{\mathbf{u}_2}(\mathbf{v}_3) = (0,1,1) - (0.5,0.5,0) - (\tfrac16,-\tfrac16,\tfrac13) = \left(-\tfrac23,\, \tfrac23,\, \tfrac23\right).
One projection short — forgetting the shadow on \mathbf{u}_1, say —
and \mathbf{u}_3 would no longer be perpendicular to it. With both
subtracted, \mathbf{u}_3 comes out perpendicular to
both \mathbf{u}_1 and
\mathbf{u}_2 — check it yourself with a couple of dot products.
- Turns any basis into an orthonormal basis of the same span.
- Each new \mathbf{u}_k is \mathbf{v}_k
minus its projection onto all previous \mathbf{u}_j.
- It factors a matrix as A = QR, with
Q orthonormal and R upper-triangular.
Run Gram–Schmidt on the columns of a matrix A and something tidy falls
out. Collect the orthonormal vectors as the columns of Q, and the
projection coefficients you subtracted as an upper-triangular R, and
you get A = QR. This QR decomposition is one of the
workhorses of numerical linear algebra — it's how computers solve
least-squares
problems stably, and part of how they find eigenvalues.
-
Order matters. Processing the vectors in a different order gives a
different orthonormal basis — same span, different vectors. Gram–Schmidt is not
unique; the first vector's direction is always kept exactly, and everything after it bends
around that choice.
-
Subtract the projection onto every previously-fixed vector, not just the last
one. Miss one and \mathbf{u}_k won't be perpendicular to all of them.
-
Don't forget the final normalise step — orthogonal is not the same as
orthonormal until each vector has length 1.
-
A zero leftover is a diagnosis, not a bug. If some
\mathbf{u}_k comes out as the zero vector, every direction it had
was already covered by the earlier ones — the original list was
linearly
dependent all along, and Gram–Schmidt just caught it for free.
A 3-D character or camera needs three mutually perpendicular axes — right,
up, and forward — to know which way it's facing. Add up small rotations frame
after frame, though, and floating-point rounding slowly nudges those three axes out of true:
"up" stops being quite perpendicular to "forward," and the character's view subtly warps.
The fix is Gram–Schmidt, run once a frame: keep forward fixed, subtract its projection
from the drifting up vector to force a right angle again, then take the cross product
of the two to rebuild a perfectly perpendicular right. Three vectors go in slightly
skewed, three come out exactly orthonormal — a tiny piece of linear algebra quietly running
every frame so the world never looks like it's leaning.