Differencing

Decomposition estimates a trend and subtracts it. Differencing does something more brutal and more elegant: it removes a trend without ever estimating it. Instead of asking "what is the level?" it asks only "how much did the series change?" — and change is what is left when a steady march is subtracted away. This one operation is the beating heart of the modern ARIMA approach, the "I" (for Integrated) in the name.

The first difference is just consecutive change:

\nabla y_t = y_t - y_{t-1}.

The operator \nabla (nabla) turns a series of levels into a series of steps. It is the discrete cousin of the derivative — where calculus takes an instantaneous rate of change, differencing takes the one-step change.

First difference kills a linear trend

Here is the magic in one line. Suppose the series is a straight trend plus noise, y_t = a + b\,t + \varepsilon_t. Then

\nabla y_t = (a + bt + \varepsilon_t) - (a + b(t-1) + \varepsilon_{t-1}) = b + (\varepsilon_t - \varepsilon_{t-1}).

The intercept a cancels and the growing term b\,t collapses to a constant b — the slope. A tilted, non-stationary line becomes a flat series wobbling around a fixed level. The trend is gone, and we never had to fit it.

First difference \nabla y_t removes a linear (degree-1) trend, leaving a constant plus noise.
Second difference \nabla^2 y_t = \nabla(\nabla y_t) removes a quadratic (degree-2) trend — differencing twice kills a parabola, just as differentiating twice kills a quadratic.
In general, a degree-d polynomial trend is annihilated by d differences.

Second differencing expands neatly: \nabla^2 y_t = y_t - 2y_{t-1} + y_{t-2} — the binomial coefficients 1, -2, 1 are no accident; the d-th difference carries the signed coefficients of (1-B)^d, where B is the backshift operator.

Seasonal differencing

Trends are not the only thing that makes a series non-stationary; a strong repeating season does too. For that there is the seasonal difference, which subtracts the value one full period ago:

\nabla_s y_t = y_t - y_{t-s}.

With s = 12 this is "this month minus the same month last year" — the year-on-year change reported in every economics headline. If the seasonal pattern repeats exactly, then y_t and y_{t-s} share it and the subtraction wipes it out, leaving the non-seasonal movement. Real series often need both: a seasonal difference to kill the season and a first difference to kill the trend, giving \nabla \nabla_{s}\, y_t — the standard pre-treatment for a seasonal ARIMA model.

Worked example — difference a curved series

Take y = (2,\ 4,\ 7,\ 11,\ 16). First differences are consecutive gaps:

\nabla y = (4-2,\ 7-4,\ 11-7,\ 16-11) = (2,\ 3,\ 4,\ 5).

Not flat — because this series is curving (the gaps themselves grow). The first difference turned a quadratic-ish climb into a straight line. Difference once more:

\nabla^2 y = (3-2,\ 4-3,\ 5-4) = (1,\ 1,\ 1).

A perfect constant — the series was exactly quadratic (y_t = \tfrac12 t^2 + \tfrac32 t + 2 for t = 0,\dots,4), so two differences flatten it completely, matching the theorem. Each round of differencing peels off one degree of polynomial trend.

Seeing a trend flatten

The top curve is a rising series with noise; the bottom curve is its first difference \nabla y_t, hovering around a constant with the climb removed. The upward march has become a level band — the visual signature that a single difference has done its job. If the differenced series still trended, you would difference again.

Checking that the differenced series looks stationary — no trend, stable spread — is exactly how you decide how many differences a series needs, made formal by the unit-root (Dickey–Fuller) test.

The random walk y_t = y_{t-1} + \varepsilon_t is the poster child for differencing. It is aggressively non-stationary — its variance grows without bound and it wanders off forever — yet its first difference is simply \nabla y_t = \varepsilon_t, pure white noise. One difference converts the most stubborn non-stationary series in the subject into the most benign stationary one. This is why so much financial data is analysed in differences (returns) rather than levels (prices): prices behave like a random walk, but returns are close to stationary.

If one difference is good, two must be better? No — over-differencing is a classic mistake with two nasty consequences. First, it inflates the variance: every extra difference adds noise (recall \nabla \varepsilon_t = \varepsilon_t - \varepsilon_{t-1} has variance 2\sigma^2, double the original). Second, it manufactures artificial structure: differencing already-stationary white noise introduces a spurious MA(1) component with a negative lag-1 autocorrelation of -0.5 that was never in the data — a unit root in the MA part, which also breaks invertibility. The rule: difference just enough to reach stationarity and not one step more. A tell-tale sign you have gone too far is a differenced series whose lag-1 autocorrelation has plunged strongly negative.