Orthogonal Functions

Two vectors are orthogonal when their dot product is zero — they point in genuinely independent directions. The whole of Fourier analysis rests on one bold move: treat functions as vectors and ask the very same question.

For vectors the dot product sums a product over the components, \mathbf{u}\cdot\mathbf{v} = \sum_i u_i v_i. A function has a continuum of "components" — its values f(x) — so the sum becomes an integral. On the interval [-L, L] we define the inner product

\langle f, g\rangle = \int_{-L}^{L} f(x)\,g(x)\,dx.

Two functions are orthogonal on that interval when \langle f, g\rangle = 0: their product has exactly as much area above the axis as below.

Why bother? Because orthogonal directions are independent — moving along one never secretly nudges you along another. A compass's north-south axis and its east-west axis are orthogonal: walking due east changes nothing about how far north you are. That independence is precisely what we want from a set of building-block functions — we'd like to describe a wiggly curve as "3 parts of this wave, plus 5 parts of that wave," with each part contributing its own piece and no others getting tangled up in the bookkeeping.

Sines and cosines are mutually orthogonal

The miracle Fourier exploited is that the functions \sin\frac{n\pi x}{L} and \cos\frac{n\pi x}{L}, for n = 1, 2, 3, \dots, are all orthogonal to one another. Over [-\pi, \pi] (taking L = \pi):

\int_{-\pi}^{\pi}\sin(mx)\sin(nx)\,dx = \begin{cases}0 & m \neq n,\\[2pt] \pi & m = n,\end{cases}

and the same pattern holds for cosines, while every sine is orthogonal to every cosine. The reason is a product-to-sum identity: \sin(mx)\sin(nx) = \tfrac12[\cos((m-n)x) - \cos((m+n)x)]. Each cosine on the right integrates to zero over a whole number of periods — unless m = n, where the first term collapses to the constant \tfrac12 and contributes area \pi.

See the cancellation

Below is the product \sin(mx)\sin(nx) on [-\pi, \pi]. Move m and n: whenever they differ, the curve has equal area above and below the axis, so its integral is zero — orthogonal. Set m = n and the curve sits mostly above the axis (it is \sin^2, never negative for the bulk), giving positive area — the functions are no longer orthogonal to themselves.

On [-L, L], for positive integers m, n:

So the sines and cosines form a set of mutually perpendicular "axes" for functions.

Worked example: check three pairs by hand

Let's verify orthogonality directly on the interval [0, 2\pi] instead of [-\pi, \pi] — it's still one full period, so the same theorem applies either way. (The theorem only cares that you integrate over a complete period; where that period starts is irrelevant, since sine and cosine simply repeat.)

1. Are \sin x and \cos x orthogonal? Using \sin x\cos x = \tfrac12\sin(2x):

\int_0^{2\pi}\sin x\cos x\,dx = \tfrac12\int_0^{2\pi}\sin(2x)\,dx = \tfrac12\Big[-\tfrac{\cos(2x)}{2}\Big]_0^{2\pi} = 0.

Yes — over a whole wave, the positive and negative lobes of the product cancel exactly.

2. Are \sin x and \sin 2x orthogonal? The product-to-sum identity gives \sin x\sin 2x = \tfrac12[\cos x - \cos 3x], and both cosines integrate to zero over a whole number of their own periods:

\int_0^{2\pi}\sin x\sin 2x\,dx = \tfrac12\Big[\sin x - \tfrac{\sin 3x}{3}\Big]_0^{2\pi} = 0.

Orthogonal again — a fundamental and its harmonic simply never "see" each other.

3. What about \sin x against itself? Now m = n, and the cancellation trick breaks down on purpose:

\int_0^{2\pi}\sin x\sin x\,dx = \int_0^{2\pi}\sin^2 x\,dx = \pi.

This self-overlap, \langle f, f\rangle, is never zero for a nonzero function — it plays the role of a vector's squared length, \mathbf{v}\cdot\mathbf{v} = |\mathbf{v}|^2. It's exactly the number you'd divide by to normalize a basis function to "unit length."

The payoff: pulling out one coefficient at a time

Suppose a periodic function is a big sum of these basis waves, f(x) = \sum_k c_k \phi_k(x). Multiply both sides by one particular \phi_n and integrate over the period:

\langle f, \phi_n\rangle = \sum_k c_k \langle \phi_k, \phi_n\rangle = c_n \langle \phi_n,\phi_n\rangle,

because orthogonality kills every term except the one where k = n — every other coefficient simply vanishes from the sum. This "project, integrate, divide by the self-overlap" recipe is exactly how the Fourier coefficients formula is built.

Compare this to ordinary vectors: to find how much of a vector \mathbf{v} points along the x-axis, you dot it with the unit vector \hat{\mathbf{x}} — the other axes contribute nothing to that dot product because they're orthogonal to \hat{\mathbf{x}}. Extracting a Fourier coefficient is the exact same manoeuvre, just with an integral standing in for the dot product and a wave standing in for an axis.

It's tempting to picture orthogonal functions as two graphs crossing at a right angle, like perpendicular arrows. Don't. Orthogonality here is a statement about an integral, not a geometric angle you could measure with a protractor on the graph paper. \sin x and \cos x are orthogonal even though their graphs are two ordinary wavy curves that cross each other many times, at all sorts of angles — some steep, some shallow. The "perpendicular" language is a powerful analogy for how the algebra behaves (independence, projections, a length-squared), but the object itself is a function, not an arrow, and it never literally points anywhere. If you find yourself hunting for a 90° angle on a graph, you've slipped back into the vector picture — step back to the integral instead.

Sines and cosines are just the most famous orthogonal family — they're far from the only one. Legendre polynomials are orthogonal on [-1, 1] and show up wherever spherical symmetry does, from gravitational fields to antenna design. Chebyshev polynomials are orthogonal under a particular weighting and are prized in numerical analysis for how evenly they approximate curves. And in quantum mechanics, the wavefunctions describing an electron's possible energy states in an atom are yet another orthogonal family — "orthogonal" there means two states are perfectly distinguishable from one another; measuring an electron in one energy state can never be confused with measuring it in another, for exactly the same reason a sine wave's coefficient never leaks into a cosine's. Wildly different maths, same one trick underneath: build a family of mutually orthogonal functions, and any function in that world becomes a simple sum of independent pieces.