The Random Walk

Take the memoryless atom — white noise — and do the most innocent thing imaginable: keep a running total. Each step, add a fresh random shock to where you already are. That single act of accumulation produces the random walk, and with it the archetype of everything that refuses to be stationary. Stock prices, the position of a pollen grain jittering in water, the cumulative winnings of a fair gambler — all are random walks, and all share the same unsettling property: they wander off and never come home.

The random walk is the hinge of the whole course. It is the boundary case between stationary and non-stationary, the \phi = 1 edge of the autoregressive world, and the reason an entire branch of the subject — differencing, unit roots, integration — exists at all.

The definition, and why it never forgets

A random walk is defined by the recursion

X_t = X_{t-1} + \varepsilon_t, \qquad \varepsilon_t \sim \text{WN}(0, \sigma^2),

starting from X_0 = 0. Unrolling the recursion shows what it really is — a cumulative sum of shocks:

X_t = \varepsilon_1 + \varepsilon_2 + \dots + \varepsilon_t = \sum_{i=1}^{t} \varepsilon_i.

Every past shock is in there at full weight, forever — nothing decays. That is the crucial contrast with a stationary AR(1), where each value is a fraction \phi of the last and old shocks fade geometrically. Here the multiplier is exactly 1: a kick received at time 3 is still fully present at time 3000. The random walk has perfect, permanent memory.

Why it is not stationary

The mean behaves innocently — but the variance gives the game away. Taking expectations of the cumulative sum:

Mean: \mathbb{E}[X_t] = \sum_{i=1}^t \mathbb{E}[\varepsilon_i] = 0 — constant, so the mean condition holds.
Variance: since the shocks are uncorrelated, \operatorname{Var}(X_t) = \sum_{i=1}^t \operatorname{Var}(\varepsilon_i) = t\,\sigma^2 — it grows without bound, linearly in time. The constant-variance condition fails.
Covariance: for s \le t, \operatorname{Cov}(X_s, X_t) = s\,\sigma^2 — it depends on the absolute time s, not merely the lag. The lag-only condition fails too.

So the random walk breaks two of the three stationarity conditions. The spread of where it might be fans out like \sqrt{t}\,\sigma (the standard deviation), which is exactly the widening cone you see when you overlay many paths.

The fanning cone

Below are five independent random walks, all starting at zero, all driven by the same kind of white-noise shocks. Early on they cling together; as time passes they spread ever wider apart. The dashed curves trace \pm \sqrt{t}\,\sigma — the one-standard-deviation envelope that the paths fill out. This cone is the visual signature of non-stationarity: a stationary series would stay inside a fixed horizontal band forever, but the random walk's uncertainty keeps growing.

Read the forecasting moral straight off the picture: from any point, the best guess for the future is simply the last value (the shocks ahead average to zero), and the prediction interval around that guess widens like \sqrt{h} as the horizon h grows — the cone, viewed from today.

The rescue: its first difference is white noise

Non-stationary though it is, the random walk is tamed by a single operation. Its first difference collapses the recursion back to the atom it was built from:

\nabla X_t = X_t - X_{t-1} = \varepsilon_t.

Differencing a random walk gives back pure white noise — perfectly stationary. This is why the random walk is called difference-stationary, or "integrated of order one," I(1): one difference makes it stationary. That single fact is the entire motivation for the "I" in ARIMA, and the boundary it sits on is what unit-root tests are built to detect.

Random walk with drift

Add a constant nudge \mu each step and you get the random walk with drift:

X_t = \mu + X_{t-1} + \varepsilon_t = \mu\,t + \sum_{i=1}^{t}\varepsilon_i.

Now there is a deterministic linear climb \mu\,t on top of the accumulating randomness — the mean is no longer constant either. This is the standard first model for a stock price or a log-price: a steady upward drift (the expected return) buried in a cloud of unpredictable accumulated shocks. It captures both the long-run trend and the wandering that makes short-run prediction hopeless — the essence of the efficient-market picture, where tomorrow's best forecast is today's price plus the drift.

Here is the trap that has wrecked countless empirical papers. Take two completely independent random walks — say a series of coin-flip cumulative sums in London and another in Tokyo, with no connection whatsoever. Regress one on the other and you will very often get a large R^2 and a wildly "significant" t-statistic, seemingly proving a strong relationship that does not exist. This is spurious regression: two independent non-stationary series both wander, and any two wandering lines will appear to move together over a finite window. The standard errors are meaningless because the residuals are themselves a non-stationary random walk, violating every assumption of ordinary regression. The fixes are to difference both series before regressing, or to test properly for a genuine long-run link — cointegration. Never trust a regression of one level-series on another until you have ruled this out.

The idea has a triple birth, which is part of why it matters so much. In 1900 the French student Louis Bachelier, in a thesis his examiners barely knew what to do with, modelled the Paris Bourse's prices as a random walk — the first mathematical theory of financial markets, decades ahead of its time. Five years later Einstein used the very same mathematics to explain Brownian motion — the jitter of pollen grains bombarded by invisible molecules — and in doing so gave physics its clinching evidence that atoms are real. And Karl Pearson coined the phrase "random walk" in 1905 posing a puzzle about a wandering drunkard in Nature. Finance, physics, and pure probability arrived at the same object almost at once. Its continuous-time limit, Brownian motion, went on to underpin the Black–Scholes option-pricing formula — so this humble running total of coin flips sits under a trillion-dollar industry.