The Gram–Schmidt Process

An orthonormal basis — vectors that are mutually perpendicular and each of length 1 — is the nicest kind of basis to work with. Coordinates come straight from dot products, with no matrix to invert. But the basis you're handed is usually skewed: the vectors span the right space, yet lean on each other at awkward angles.

The Gram–Schmidt process straightens them out. Given any basis, it produces an orthonormal basis spanning exactly the same space, by a single repeated idea: take each vector and subtract off the part that points along the directions you've already fixed, leaving only what is genuinely new and perpendicular.

The one move: subtract the projection

The engine is the projection of one vector onto another. Start with a basis \mathbf{v}_1, \mathbf{v}_2, \dots:

Keep the first direction: \mathbf{u}_1 = \mathbf{v}_1.
For each next \mathbf{v}_k, subtract its projection onto every direction already chosen:

\mathbf{u}_k = \mathbf{v}_k - \sum_{j < k} \operatorname{proj}_{\mathbf{u}_j}(\mathbf{v}_k), \qquad \operatorname{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\,\mathbf{u}.

What's left, \mathbf{u}_k, is perpendicular to all the earlier ones. Finally normalise each to length 1: \mathbf{e}_k = \mathbf{u}_k / \lVert \mathbf{u}_k \rVert, and you have an orthonormal basis.

Here is the key step in the plane: \mathbf{v}_2 minus its shadow on \mathbf{v}_1 leaves a piece exactly perpendicular to \mathbf{v}_1. Step through it:

A worked example

Take \mathbf{v}_1 = (3, 1) and \mathbf{v}_2 = (1, 2). Set \mathbf{u}_1 = (3, 1). Then

\operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_2) = \frac{\mathbf{v}_2 \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\,\mathbf{u}_1 = \frac{3 + 2}{9 + 1}\,(3,1) = \tfrac{1}{2}(3,1) = (1.5,\, 0.5),

\mathbf{u}_2 = \mathbf{v}_2 - \operatorname{proj}_{\mathbf{u}_1}(\mathbf{v}_2) = (1,2) - (1.5, 0.5) = (-0.5,\, 1.5).

Check they are perpendicular: \mathbf{u}_1 \cdot \mathbf{u}_2 = (3)(-0.5) + (1)(1.5) = -1.5 + 1.5 = 0. ✓ Normalising each gives the orthonormal pair.

Turns any basis into an orthonormal basis of the same span.
Each new \mathbf{u}_k is \mathbf{v}_k minus its projection onto all previous \mathbf{u}_j.
It factors a matrix as A = QR, with Q orthonormal and R upper-triangular.

Run Gram–Schmidt on the columns of a matrix A and something tidy falls out. Collect the orthonormal vectors as the columns of Q, and the projection coefficients you subtracted as an upper-triangular R, and you get A = QR. This QR decomposition is one of the workhorses of numerical linear algebra — it's how computers solve least-squares problems stably, and part of how they find eigenvalues.

Order matters. Processing the vectors in a different order gives a different orthonormal basis (still valid, just different) — the first vector's direction is always kept.
Subtract the projection onto every previously-fixed vector, not just the last one. Miss one and \mathbf{u}_k won't be perpendicular to all of them.
Don't forget the final normalise step — orthogonal is not the same as orthonormal until each vector has length 1.