The Spectral Density
Play a chord on a piano and a microphone records a single wobbling voltage — one number per instant, a
textbook time series. Yet your ear does not hear "a wobbling voltage": it hears three notes at
once. Somewhere between the air and the auditory cortex, the signal gets re-sorted from a story
about time ("what is the value now?") into a story about frequency
("how much of each pitch is in the mix?"). That change of viewpoint — same data, rotated ninety degrees —
is the whole of spectral analysis, and the object that carries it is the
spectral density.
Everything you already know about a stationary series lives in its
autocovariance
function \gamma(h) — how a value co-varies with its own past
h steps back. The spectral density is that same information written in
the language of frequencies. Nothing is added and nothing is lost; we merely choose a basis in which
certain questions become easy.
From autocovariance to spectrum
Take a zero-mean stationary process with autocovariances \gamma(h). Its
spectral density is the Fourier
transform of that sequence:
f(\omega) = \frac{1}{2\pi}\sum_{h=-\infty}^{\infty} \gamma(h)\, e^{-i\omega h},
\qquad -\pi \le \omega \le \pi .
The frequency \omega is in radians per time step: a slow
drift lives near \omega = 0, the fastest wiggle a sampled series can even
represent lives at \omega = \pi (one full flip every two observations). By
Euler's formula,
e^{-i\omega h} = \cos(\omega h) - i\sin(\omega h); because
\gamma(h)=\gamma(-h) is symmetric, the imaginary parts cancel in pairs and
f(\omega) comes out real and non-negative — a genuine density over
frequency.
- For a stationary process the spectral density and the autocovariance function are a
Fourier transform pair:
f(\omega) = \tfrac{1}{2\pi}\sum_h \gamma(h)e^{-i\omega h}.
- The relationship inverts:
\gamma(h) = \displaystyle\int_{-\pi}^{\pi} f(\omega)\,e^{i\omega h}\,d\omega.
- Setting h=0 gives the variance decomposition
\int_{-\pi}^{\pi} f(\omega)\,d\omega = \gamma(0) = \operatorname{Var}(x_t):
the total power of the series, sliced up frequency by frequency.
That last line is the punchline. The spectral density is a recipe for the series' variance: the area
under f over a band of frequencies is exactly the variance contributed by
oscillations in that band. Fast structure and slow structure each get their share, and the shares sum to
the total.
Why noise is called "white"
Take the simplest series of all,
white noise: mean
zero, variance \sigma^2, and no serial dependence, so
\gamma(0)=\sigma^2 and \gamma(h)=0 for
h\neq 0. Only one term survives the sum:
f(\omega) = \frac{\sigma^2}{2\pi}, \qquad \text{a flat line for every } \omega .
Every frequency carries the same power — exactly as white light contains every colour in equal
measure. That is where the name comes from, and it runs the other way too: a process with a flat
spectrum is white noise. Structure in a series always shows up as a spectrum that is
not flat — humped, tilted, or spiked.
Red, blue, and a peak: the AR(1) spectrum
A first-order autoregression,
AR(1)
x_t = \phi x_{t-1} + \varepsilon_t, has autocovariance
\gamma(h) = \sigma^2 \phi^{|h|}/(1-\phi^2). Feed that geometric sequence into
the sum and it collapses to a clean closed form:
f(\omega) = \frac{\sigma^2}{2\pi}\cdot\frac{1}{\,1 - 2\phi\cos\omega + \phi^2\,}.
Drag \phi and watch where the power piles up (here
\sigma^2 = 1):
- \phi > 0 — a "red" spectrum. Power concentrates at
low frequencies (near \omega = 0): the series is smooth and
drifts in long, slow swings, like red light sitting at the low-frequency end.
- \phi < 0 — a "blue" spectrum. Power piles up at
high frequencies (near \omega = \pi): the series
zig-zags, flipping sign almost every step.
The peak matters. A hump in the spectrum at some \omega_0 announces a
dominant period of 2\pi/\omega_0 time steps — the series
"prefers" to oscillate at that rate. A peak at \omega_0 = \pi/6, say, means a
favoured cycle of length 12; find such a peak in monthly data and you have
found an annual season, purely from the shape of the spectrum.
A worked example: splitting the variance
Suppose x_t = \tfrac{1}{2}x_{t-1} + \varepsilon_t with
\sigma^2 = 1. Its variance is
\gamma(0) = \sigma^2/(1-\phi^2) = 1/(1-0.25) = 4/3 \approx 1.33. The
spectral density is f(\omega) = \frac{1}{2\pi}\cdot\frac{1}{1.25 - \cos\omega}.
Evaluate it at the ends:
- At \omega = 0:
f(0) = \frac{1}{2\pi}\cdot\frac{1}{0.25} = \frac{2}{\pi} \approx 0.637 —
the tall low-frequency shoulder.
- At \omega = \pi:
f(\pi) = \frac{1}{2\pi}\cdot\frac{1}{2.25} \approx 0.0707 — the low
high-frequency floor.
Low frequencies carry roughly nine times the power of high ones: this series is decisively "red". And if
you integrated f across the whole band from
-\pi to \pi, you would recover exactly
1.33 — the variance, whole again.
No new information — and that is the point, not a flaw. The spectral density and the autocovariance are
two encodings of the identical content, related by an invertible Fourier transform; you can compute
either from the other and lose nothing. What changes is which questions are cheap. "Is there a
cycle of length 12?" is a nightmare to read off a wiggly ACF but jumps out as a peak in the spectrum.
"How fast does memory decay?" is obvious from the ACF and awkward in the spectrum. Convolutions in time
become plain multiplications in frequency — which is why running a series through a
linear
filter is far easier to reason about in the frequency domain. Same facts, better-shaped tools.
A common beginner's mistake is to treat "time-domain analysis" (ACF, AR/MA models) and "frequency-domain
analysis" (the spectrum) as two competing schools you must pick between, as if one might reveal a cycle
the other misses. They cannot disagree: they are a Fourier transform pair of the very
same \gamma(h). If a periodicity is genuinely in the process, it is in
both descriptions — a slowly-tapering, gently-oscillating ACF and a spectral peak are the same
statement said twice. Choose the domain that makes your question easy; never imagine you are
choosing between different truths.
Joseph Fourier was studying how heat spreads through a
metal bar when he made an outrageous claim: any reasonable function can be built as a sum of
sines and cosines. Contemporaries scoffed; the idea turned out to underpin signal processing, quantum
mechanics, image compression and — here — the spectrum of a time series. The autocovariance
\gamma(h) is just such a function of the lag h, and
the spectral density is its Fourier partner. Every time you decompose a series into frequencies you are
replaying Fourier's heat-bath insight from 1807.