Prediction Intervals

A point forecast that says "sales next quarter will be 4.2 million units" is, on its own, almost useless — and quietly dishonest. Will it be 4.2 give-or-take a rounding error, or give-or-take three million? A forecast without a stated uncertainty is a number pretending to be a fact. The remedy is the prediction interval: a range, like 4.2 \pm 0.9, that we expect to contain the realised value with a stated probability (usually 95%). This page is about where that width comes from and why it grows the further ahead you look.

Forecast error, written as a sum of shocks

The interval is built around the forecast error e_{T+h} = x_{T+h} - \hat x_{T+h}. Write any stationary ARMA in its MA(∞) form as a weighted sum of shocks, x_t = \sum_{j \ge 0} \psi_j\, \varepsilon_{t-j} (the \psi_j are the impulse-response weights). The optimal forecast uses every shock up to time T; the error is therefore exactly the shocks that hadn't happened yet when we forecast:

e_{T+h} = \sum_{j=0}^{h-1} \psi_j\, \varepsilon_{T+h-j}.

These future shocks are independent, mean-zero, variance \sigma^2. So the error has mean zero (the forecast is unbiased) and a variance that is just the sum of the squared weights:

The variance of the h-step forecast error is \sigma_h^2 = \operatorname{Var}(e_{T+h}) = \sigma^2 \sum_{j=0}^{h-1} \psi_j^2.
It is non-decreasing in h: every extra step adds another non-negative term \sigma^2 \psi_{h}^2.
The one-step error variance is always \sigma_1^2 = \sigma^2 (only \psi_0 = 1 contributes).

From variance to interval

If the shocks are Gaussian, the forecast error is normal with standard deviation \sigma_h, and the interval is the familiar bell-curve band:

\hat x_{T+h} \;\pm\; 1.96\,\sigma_h \qquad (\text{95\% prediction interval}).

The 1.96 is the 97.5th percentile of the standard normal; swap it for 1.645 to get a 90% interval, or 2.576 for 99%. Because \sigma_h grows with the horizon, the intervals fan out: a narrow throat at h = 1 flaring into a wide cone far ahead. The chart shows the point forecast decaying to the mean while the \pm 1.96\sigma_h bands widen around it — the signature "trumpet" of every forecast plot.

Worked example — the AR(1) interval

For an AR(1) the MA weights are \psi_j = \phi^{\,j}, so the error-variance sum is a truncated geometric series:

\sigma_h^2 = \sigma^2\big(1 + \phi^2 + \phi^4 + \dots + \phi^{2(h-1)}\big) = \sigma^2\,\frac{1 - \phi^{2h}}{1 - \phi^2}.

One step: \sigma_1 = \sigma — the interval half-width is just 1.96\,\sigma.
Two steps: \sigma_2 = \sigma\sqrt{1 + \phi^2} — wider by a factor \sqrt{1+\phi^2}.
Far ahead: \sigma_h \to \sigma/\sqrt{1-\phi^2}, the unconditional standard deviation — the cone stops flaring and settles at the natural spread of the series.

With \phi = 0.6 and \sigma = 1: the 1-step band is \pm 1.96, the 2-step band \pm 1.96\sqrt{1.36} \approx \pm 2.29, and the long-run band settles near \pm 1.96/\sqrt{0.64} = \pm 2.45.

The random-walk exception: intervals that never stop widening

For a random walk (\phi = 1) every weight is \psi_j = 1, so the sum is just h and

\sigma_h = \sigma\sqrt{h}.

The band grows without bound, in proportion to \sqrt{h} — the same \sqrt{h} that governs a diffusing gas molecule or a gambler's fortune. Unlike a stationary series, whose cone plateaus at the unconditional spread, a non-stationary series confesses ever more ignorance the further out you look. That contrast — plateauing versus endlessly widening bands — is a visual test for stationarity in disguise.

The tidy formula \hat x_{T+h} \pm 1.96\,\sigma_h quietly assumes you know the true parameters — it uses \hat\phi, \hat\theta, \hat\sigma^2 as if they were exact. They are not: they were estimated from finite data and carry their own uncertainty, which the plug-in interval simply ignores. The result is systematically over-confident intervals — a nominal "95%" band that in truth catches the outcome rather less than 95% of the time, especially at long horizons and small samples. On top of that, the interval assumes the model form is correct and the shocks are Gaussian; fat tails or a misspecified model make things worse still. Rules of thumb: prefer intervals that account for parameter uncertainty (bootstrap or Bayesian), and treat the analytic band as an optimistic lower bound on the true width.

The 1.96 is nothing more than "how many standard deviations out you go to capture the middle 95% of a normal curve" — it is \Phi^{-1}(0.975). It is exactly right only when the forecast error is Gaussian, which holds when the shocks are Gaussian (a sum of normals is normal). If the shocks have heavier tails, the \pm 1.96\sigma_h band is still a decent 95% interval for large h — the error is a sum of many shocks, and the central limit theorem nudges it back toward normal — but for short horizons, where the error is a sum of just one or two fat-tailed shocks, the true interval can be markedly wider than the Gaussian formula admits. When tails matter (finance, especially), model them directly with a richer error model rather than trusting the 1.96.