Diagonalization
If a matrix has a full set of independent
eigenvectors,
something wonderful happens: in the
basis made
of those eigenvectors, the matrix acts as a plain
diagonal
stretch. We can write
A = P D P^{-1},
where the columns of P are the eigenvectors and
D is the diagonal matrix of eigenvalues. Read right to left:
P^{-1} rewrites a vector in eigen-coordinates,
D stretches along each eigen-axis, and
P translates back. The tangled matrix was a simple stretch all along —
just along the wrong-looking axes.
A stretch along its own axes
The faint grid is aligned to the eigenvectors of
A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}. Drag the slider to
apply A: the grid doesn't rotate or twist — each eigen-axis simply
stretches by its eigenvalue (×3 along one, ×1 along the other). In its own basis,
A is nothing but two scalings.
Why diagonal is gold
Diagonal matrices are trivial to raise to powers — and so are diagonalizable ones, because
A^n = P D^n P^{-1} just powers the eigenvalues. That single trick
evaluates matrix powers instantly, solves systems of differential equations, computes the
long-run behaviour of
Markov processes,
and sets up the
decompositions
that data science leans on. Eigenvectors turn "apply this matrix a thousand times" into "raise
two numbers to the thousandth power."