The Spectral Theorem

Most matrices, when you diagonalise them, give you an eigenvector basis that is skewed — the change-of-basis matrix P is some awkward invertible thing you must carefully invert. Symmetric matrices are gloriously different. The spectral theorem says that a real symmetric matrix always has a perfect eigenbasis: its eigenvectors can be chosen orthonormal, and its eigenvalues are all real.

You have already seen the seed of this: for a symmetric matrix, eigenvectors from different eigenvalues are automatically perpendicular. The spectral theorem is the full, guaranteed conclusion.

Orthogonal diagonalisation

Because the eigenvectors are orthonormal, the matrix Q holding them as columns is orthogonal — its inverse is just its transpose, Q^{-1} = Q^{\mathsf{T}}. So diagonalisation stops needing any messy inverse:

Every real symmetric matrix A = A^{\mathsf{T}} has real eigenvalues.
Its eigenvectors can be chosen to form an orthonormal basis.
Hence A = Q \Lambda Q^{\mathsf{T}}, with Q orthogonal (Q^{\mathsf{T}}Q = I) and \Lambda the diagonal matrix of eigenvalues.

Read A = Q \Lambda Q^{\mathsf{T}} geometrically and it is beautiful: every symmetric matrix acts by rotating to its eigenaxes (Q^{\mathsf{T}}), stretching along them by the eigenvalues (\Lambda), and rotating back (Q). No shearing, ever — just scaling along a set of perpendicular axes.

A worked example

Take the symmetric matrix

A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}.

Its eigenvalues are \lambda_1 = 3 and \lambda_2 = 1 (both real), with eigenvectors (1, 1) and (1, -1) — which are perpendicular: (1)(1) + (1)(-1) = 0. Normalise them to length 1 and stack as columns:

Q = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}, \qquad \Lambda = \begin{bmatrix} 3 & 0 \\ 0 & 1 \end{bmatrix}, \qquad A = Q\,\Lambda\,Q^{\mathsf{T}}.

The ellipse below is the unit circle transformed by A: it stretches by 3 along the (1,1) eigen-axis and by 1 along the perpendicular (1,-1) axis — the eigenaxes are exactly the axes of the ellipse.

Symmetric matrices are everywhere real quantities pair up symmetrically: a quadratic form, a covariance matrix in statistics, a moment-of-inertia tensor in mechanics, the Hessian of second derivatives. The spectral theorem promises each has clean perpendicular "principal axes" and real eigenvalues — which is exactly what principal component analysis finds when it decomposes a covariance matrix.

The clean guarantee is for real symmetric matrices. A general matrix may have complex eigenvalues or non-orthogonal (even missing) eigenvectors.
Orthogonal eigenvectors are automatic only across different eigenvalues. For a repeated eigenvalue you can still choose an orthonormal basis of its eigenspace (via Gram–Schmidt) — the theorem guarantees one exists.