Linear Filters and Frequency Response

Every graphic equaliser, every "smooth the noisy sensor" line of code, every seasonal adjustment an economic agency publishes, is a linear filter: a machine that takes a series in and puts a reshaped series out by forming weighted combinations of the input. The bass-and-treble knobs on a stereo are the whole idea in furniture form — turn one band up, another down, leave the rest alone. The deep question of this lesson is: which knob does a given filter turn? The answer lives entirely in the frequency domain, and it is astonishingly clean.

The filter and its frequency response

A (time-invariant) linear filter forms

y_t = \sum_{j} a_j\, x_{t-j},

with fixed weights a_j. The moving-average smoother is exactly this, with equal weights. Now feed in a single complex sinusoid x_t = e^{i\omega t}. Out comes y_t = \big(\sum_j a_j e^{-i\omega j}\big)e^{i\omega t} — the same frequency, merely rescaled by a complex number. That number is the filter's frequency-response (transfer) function:

A(\omega) = \sum_{j} a_j\, e^{-i\omega j}.

Sinusoids are the filter's eigenfunctions: a linear filter can never create a new frequency, only scale and shift the ones already present. Write A(\omega) in polar form A(\omega) = |A(\omega)|\,e^{-i\phi(\omega)} and its two parts have names:

Gain |A(\omega)| — how much the amplitude at frequency \omega is boosted or cut.
Phase \phi(\omega) — how far that component is shifted in time (a delay).

What a filter does to the spectrum

Because the filter scales each frequency by A(\omega), it scales each frequency's power by |A(\omega)|^2. That is the master identity of the whole subject:

If y_t = \sum_j a_j x_{t-j} then the output spectrum is the input spectrum times the squared gain: f_y(\omega) = |A(\omega)|^2\, f_x(\omega).
The messy time-domain convolution becomes a plain multiplication in frequency — the reason spectral thinking makes filters easy.
The function |A(\omega)|^2 is the filter's power transfer function: read straight off it which bands survive and which are crushed.

So designing a filter is designing a shape for |A(\omega)|^2. Three archetypes:

Low-pass. Passes low frequencies, attenuates high ones — a smoother. The moving average is the canonical example.
High-pass. Passes high frequencies, kills the slow drift and the mean — differencing y_t = x_t - x_{t-1} is the workhorse.
Band-pass. Keeps a middle band and rejects both ends — used to isolate a business cycle or a tidal constituent.

The moving average is a low-pass filter

Take the symmetric 3-term average y_t = \tfrac13(x_{t-1}+x_t+x_{t+1}). Its response is A(\omega) = \tfrac13(e^{i\omega}+1+e^{-i\omega}) = \tfrac13(1+2\cos\omega), so the power transfer is

|A(\omega)|^2 = \left(\frac{1+2\cos\omega}{3}\right)^2 .

At \omega=0 it equals 1 — a slow trend passes untouched. Toward \omega=\pi it collapses toward zero — the fast jitter is wiped out. That is smoothing, seen in the frequency domain. The 5-term average pushes the cut lower and steeper:

Notice the 3-term curve actually dips to zero at \omega = 2\pi/3 and then rises to a small bump — a side-lobe. A crude moving average does not cut cleanly; it lets a little high-frequency power leak back through, one reason careful filters use tapered weights.

Differencing is a high-pass filter

First differencing y_t = x_t - x_{t-1} has A(\omega) = 1 - e^{-i\omega} and |A(\omega)|^2 = 2 - 2\cos\omega. It is a mirror image of the smoother: 0 at \omega = 0 (it annihilates a constant and any slow trend — that is exactly why we difference to remove a trend) and a maximum of 4 at \omega = \pi (it amplifies the fastest wiggles).

Reading these two pictures together is the payoff: smoothing and differencing are not vague verbal recipes but precisely opposite shapings of the spectrum.

A worked example: filtering white noise

Pass white noise (flat spectrum f_x(\omega) = \sigma^2/2\pi) through the 3-term average. The output spectrum is f_y(\omega) = \big(\tfrac{1+2\cos\omega}{3}\big)^2\,\tfrac{\sigma^2}{2\pi} — no longer flat! It is humped at \omega=0 and near-zero at \omega=\pi. The filtered series now has a "red", low-frequency character: it looks smoother, slower, more cyclic-ish — even though the input had no structure whatsoever. Hold that thought.

A filter's output is a convolution of the input with the weight sequence a_j. The convolution theorem says the Fourier transform of a convolution is the product of the transforms — so in the frequency domain the tangled sum \sum_j a_j x_{t-j} becomes the simple product A(\omega)\,X(\omega). Squaring magnitudes to get power gives f_y = |A|^2 f_x. This is the same reason the whole of signal processing lives in frequency space: operations that are painful convolutions in time are effortless multiplications once transformed.

The worked example above is a warning in disguise. Smoothing pure noise produced a series with a low-frequency hump — and to the eye, a low-frequency hump reads as a slow, wandering cycle. This is the Slutsky–Yule effect: taking moving averages (or otherwise filtering) of structureless data can conjure smooth, apparently periodic swings out of nothing. Historically it detonated a small crisis, because many "business cycles" people had proudly extracted were partly artefacts of the smoothing used to reveal them. Two morals: (1) never read a cycle off a heavily filtered series without asking what the filter itself would do to noise; and (2) remember filters also introduce a phase shift — a one-sided moving average delays the series, so its peaks land later than the true ones. Gain tells you what survives; phase tells you how much it slid.