Stationarity

Here is the uncomfortable bargain at the heart of time-series analysis. We have one realization of a process, and we want to say something about the process — its mean, its dependence, and above all its future. But if the rules of the game are allowed to change at every instant, one path of data tells us nothing: every new observation is the first and only sample from a brand-new distribution. To learn from the past we need the past and the future to be, in a precise sense, the same kind of thing. That precise sameness is stationarity.

Intuitively, a stationary series looks statistically "the same" wherever you slide your viewing window: no drifting level, no swelling spread, the same texture of wiggle in 1990 as in 2020. It is the assumption that makes forecasting even possible — it is what lets a relationship measured yesterday be projected onto tomorrow.

Two definitions, strict and weak

There are two grades of the idea. The strong one asks that the entire probabilistic character of the series be shift-invariant.

For every set of times t_1, \dots, t_n and every shift h, the joint distribution of (X_{t_1}, \dots, X_{t_n}) equals that of (X_{t_1 + h}, \dots, X_{t_n + h}).
In words: all finite-dimensional distributions are invariant to sliding the clock. The probabilistic laws do not know what year it is.

Strict stationarity is beautiful but far more than we can ever check from data — it constrains every moment and every joint shape. So in practice we almost always work with a leaner cousin that asks only about the first two moments.

Constant mean: \mathbb{E}[X_t] = \mu for all t — no trend.
Constant, finite variance: \operatorname{Var}(X_t) = \gamma(0) < \infty for all t — the spread does not grow.
Lag-only autocovariance: \operatorname{Cov}(X_t, X_{t+h}) = \gamma(h) depends on the lag h alone, not on the absolute time t.

This is the workhorse. Because it only involves means and covariances, it is exactly the assumption a covariance function needs to be well defined, and it is the setting for essentially every model in this course. From here on, "stationary" means weakly stationary unless stated otherwise.

Strict versus weak — how they relate

Neither notion contains the other in general. Strict stationarity does not imply weak stationarity, because a strictly stationary process might have infinite variance (a heavy-tailed process is identically distributed through time yet has no finite second moment to be constant). And weak stationarity does not imply strict, because matching the first two moments says nothing about the third and beyond. The one clean bridge is the Gaussian case:

For a Gaussian process, the mean and autocovariance determine the entire joint distribution.
Hence for a Gaussian process, weak stationarity \iff strict stationarity — the two grades collapse into one.

See it move

The curve below is a deterministic stand-in for a series. With the dial at 0 it is a flat, even band oscillating around a fixed level with a fixed amplitude — the picture of stationarity. Turn the dial up and you inject a trend (the level drifts) and a growing amplitude (the variance swells): both the constant-mean and the constant-variance conditions break at once. Watch how the "same texture everywhere" quality is destroyed the moment the dial leaves zero.

A model fitted to the left third of a drifting series would badly misdescribe the right third — which is exactly why we insist on removing trend and stabilizing variance before modelling.

Worked example: two candidate series

Series A — centred noise. Let X_t = \mu + \varepsilon_t with \varepsilon_t zero-mean, variance \sigma^2, uncorrelated across time. Check the three conditions: \mathbb{E}[X_t] = \mu (constant ✓); \operatorname{Var}(X_t) = \sigma^2 (constant, finite ✓); \operatorname{Cov}(X_t, X_{t+h}) = 0 for h \neq 0, which depends only on h (✓). Weakly stationary.

Series B — linear trend. Let Y_t = a + b\,t + \varepsilon_t with the same noise and b \neq 0. Now \mathbb{E}[Y_t] = a + b\,t, which changes with t. The very first condition already fails. Not stationary — no need to check the rest. The offender is the deterministic trend, and the fix is to model or remove it.

This is the everyday diagnosis: a wandering mean or a fanning-out spread is a stationarity violation you must fix before an ARMA-type model has any right to be fitted.

Getting to stationarity

Most raw series are not stationary — and that is fine, because a small toolkit of transformations usually tames them, each targeting a specific violation:

Differencing — replace the series by its change, \nabla X_t = X_t - X_{t-1}. A linear trend has a constant difference, so one difference flattens it; this is the engine behind ARIMA models and the whole unit-root story.
Logs / power transforms — when the spread grows with the level (variance swelling as the series rises), taking logs stabilizes the variance and turns multiplicative growth into additive.
Seasonal differencing — subtract the value one period ago, X_t - X_{t-s}, to strip out a repeating seasonal pattern.

The strategy is always the same: transform the observed series into something plausibly stationary, model that, then invert the transformation to forecast the original.

A series can look non-stationary for two genuinely different reasons, and the wrong fix leaves you worse off. If the truth is Y_t = a + b\,t + \text{(stationary noise)} — a fixed line plus stationary wiggle — it is trend-stationary, and the right move is to subtract the fitted line (detrend). If instead the truth is a random walk, where the "trend" is really accumulated shocks with no fixed line at all, it is difference-stationary, and the right move is to difference. Apply the wrong remedy and you introduce artefacts: differencing a trend-stationary series injects a spurious moving-average structure, while detrending a random walk leaves a still-non-stationary residual. The two cases look nearly identical to the eye — telling them apart is precisely the job of a unit-root test.

Strictly, almost nothing is. Climate drifts, economies grow, technologies disrupt — the honest answer is that stationarity is a modelling idealization, not a fact about the universe. The pragmatic stance, due in spirit to Box, is that it is a useful fiction: real series are often stationary enough, over a chosen window, after the right transformation, for the machinery to give reliable short-horizon forecasts. The art is choosing that window and that transformation so the residual really does look like it forgot what year it is. When the world genuinely refuses to sit still, we reach for richer tools — state-space models with time-varying parameters — but even those lean on local, conditional versions of the same idea.