The Spectral Density

Play a chord on a piano and a microphone records a single wobbling voltage — one number per instant, a textbook time series. Yet your ear does not hear "a wobbling voltage": it hears three notes at once. Somewhere between the air and the auditory cortex, the signal gets re-sorted from a story about time ("what is the value now?") into a story about frequency ("how much of each pitch is in the mix?"). That change of viewpoint — same data, rotated ninety degrees — is the whole of spectral analysis, and the object that carries it is the spectral density.

Everything you already know about a stationary series lives in its autocovariance function \gamma(h) — how a value co-varies with its own past h steps back. The spectral density is that same information written in the language of frequencies. Nothing is added and nothing is lost; we merely choose a basis in which certain questions become easy.

From autocovariance to spectrum

Take a zero-mean stationary process with autocovariances \gamma(h). Its spectral density is the Fourier transform of that sequence:

f(\omega) = \frac{1}{2\pi}\sum_{h=-\infty}^{\infty} \gamma(h)\, e^{-i\omega h}, \qquad -\pi \le \omega \le \pi .

The frequency \omega is in radians per time step: a slow drift lives near \omega = 0, the fastest wiggle a sampled series can even represent lives at \omega = \pi (one full flip every two observations). By Euler's formula, e^{-i\omega h} = \cos(\omega h) - i\sin(\omega h); because \gamma(h)=\gamma(-h) is symmetric, the imaginary parts cancel in pairs and f(\omega) comes out real and non-negative — a genuine density over frequency.

That last line is the punchline. The spectral density is a recipe for the series' variance: the area under f over a band of frequencies is exactly the variance contributed by oscillations in that band. Fast structure and slow structure each get their share, and the shares sum to the total.

Why noise is called "white"

Take the simplest series of all, white noise: mean zero, variance \sigma^2, and no serial dependence, so \gamma(0)=\sigma^2 and \gamma(h)=0 for h\neq 0. Only one term survives the sum:

f(\omega) = \frac{\sigma^2}{2\pi}, \qquad \text{a flat line for every } \omega .

Every frequency carries the same power — exactly as white light contains every colour in equal measure. That is where the name comes from, and it runs the other way too: a process with a flat spectrum is white noise. Structure in a series always shows up as a spectrum that is not flat — humped, tilted, or spiked.

Red, blue, and a peak: the AR(1) spectrum

A first-order autoregression, AR(1) x_t = \phi x_{t-1} + \varepsilon_t, has autocovariance \gamma(h) = \sigma^2 \phi^{|h|}/(1-\phi^2). Feed that geometric sequence into the sum and it collapses to a clean closed form:

f(\omega) = \frac{\sigma^2}{2\pi}\cdot\frac{1}{\,1 - 2\phi\cos\omega + \phi^2\,}.

Drag \phi and watch where the power piles up (here \sigma^2 = 1):

The peak matters. A hump in the spectrum at some \omega_0 announces a dominant period of 2\pi/\omega_0 time steps — the series "prefers" to oscillate at that rate. A peak at \omega_0 = \pi/6, say, means a favoured cycle of length 12; find such a peak in monthly data and you have found an annual season, purely from the shape of the spectrum.

A worked example: splitting the variance

Suppose x_t = \tfrac{1}{2}x_{t-1} + \varepsilon_t with \sigma^2 = 1. Its variance is \gamma(0) = \sigma^2/(1-\phi^2) = 1/(1-0.25) = 4/3 \approx 1.33. The spectral density is f(\omega) = \frac{1}{2\pi}\cdot\frac{1}{1.25 - \cos\omega}. Evaluate it at the ends:

Low frequencies carry roughly nine times the power of high ones: this series is decisively "red". And if you integrated f across the whole band from -\pi to \pi, you would recover exactly 1.33 — the variance, whole again.

No new information — and that is the point, not a flaw. The spectral density and the autocovariance are two encodings of the identical content, related by an invertible Fourier transform; you can compute either from the other and lose nothing. What changes is which questions are cheap. "Is there a cycle of length 12?" is a nightmare to read off a wiggly ACF but jumps out as a peak in the spectrum. "How fast does memory decay?" is obvious from the ACF and awkward in the spectrum. Convolutions in time become plain multiplications in frequency — which is why running a series through a linear filter is far easier to reason about in the frequency domain. Same facts, better-shaped tools.

A common beginner's mistake is to treat "time-domain analysis" (ACF, AR/MA models) and "frequency-domain analysis" (the spectrum) as two competing schools you must pick between, as if one might reveal a cycle the other misses. They cannot disagree: they are a Fourier transform pair of the very same \gamma(h). If a periodicity is genuinely in the process, it is in both descriptions — a slowly-tapering, gently-oscillating ACF and a spectral peak are the same statement said twice. Choose the domain that makes your question easy; never imagine you are choosing between different truths.

Joseph Fourier was studying how heat spreads through a metal bar when he made an outrageous claim: any reasonable function can be built as a sum of sines and cosines. Contemporaries scoffed; the idea turned out to underpin signal processing, quantum mechanics, image compression and — here — the spectrum of a time series. The autocovariance \gamma(h) is just such a function of the lag h, and the spectral density is its Fourier partner. Every time you decompose a series into frequencies you are replaying Fourier's heat-bath insight from 1807.