Orthogonal Functions

Two vectors are orthogonal when their dot product is zero — they point in genuinely independent directions. The whole of Fourier analysis rests on one bold move: treat functions as vectors and ask the very same question.

For vectors the dot product sums a product over the components, \mathbf{u}\cdot\mathbf{v} = \sum_i u_i v_i. A function has a continuum of "components" — its values f(x) — so the sum becomes an integral. On the interval [-L, L] we define the inner product

\langle f, g\rangle = \int_{-L}^{L} f(x)\,g(x)\,dx.

Two functions are orthogonal on that interval when \langle f, g\rangle = 0: their product has exactly as much area above the axis as below.

Sines and cosines are mutually orthogonal

The miracle Fourier exploited is that the functions \sin\frac{n\pi x}{L} and \cos\frac{n\pi x}{L}, for n = 1, 2, 3, \dots, are all orthogonal to one another. Over [-\pi, \pi] (taking L = \pi):

\int_{-\pi}^{\pi}\sin(mx)\sin(nx)\,dx = \begin{cases}0 & m \neq n,\\[2pt] \pi & m = n,\end{cases}

and the same pattern holds for cosines, while every sine is orthogonal to every cosine. The reason is a product-to-sum identity: \sin(mx)\sin(nx) = \tfrac12[\cos((m-n)x) - \cos((m+n)x)]. Each cosine on the right integrates to zero over a whole number of periods — unless m = n, where the first term collapses to the constant \tfrac12 and contributes area \pi.

See the cancellation

Below is the product \sin(mx)\sin(nx) on [-\pi, \pi]. Move m and n: whenever they differ, the curve has equal area above and below the axis, so its integral is zero — orthogonal. Set m = n and the curve sits mostly above the axis (it is \sin^2, never negative for the bulk), giving positive area — the functions are no longer orthogonal to themselves.

On [-L, L], for positive integers m, n:

So the sines and cosines form a set of mutually perpendicular "axes" for functions.