Causality and Invertibility

Writing down \phi(B)x_t = \theta(B)\varepsilon_t is easy. Making sure it describes a sensible, unique process is the subtle part. Two things can go wrong. First, the equation might implicitly demand that today's value depend on the future — a physically absurd model that fits the data just as well as a sane one. Second, two completely different parameter sets can produce the identical autocorrelation structure, so the data cannot tell them apart. The twin conditions of causality and invertibility rule out both pathologies, and — beautifully — each reduces to a single geometric check: where do the roots of a polynomial sit relative to the unit circle?

Causality: only the past may act

A model is causal (sometimes "stationary and causal") if the present can be written as a convergent weighted sum of present and past shocks only — never future ones:

x_t = \sum_{j=0}^{\infty} \psi_j\,\varepsilon_{t-j}, \qquad \sum_{j=0}^{\infty} |\psi_j| < \infty.

This is the linear-process form again, obtained by "dividing" the ARMA equation: x_t = \big(\theta(B)/\phi(B)\big)\varepsilon_t. That division produces a convergent series precisely when the AR polynomial's roots avoid the unit circle from outside.

An ARMA process is causal if and only if every root z of the AR polynomial \phi(z) = 1 - \phi_1 z - \dots - \phi_p z^{p} = 0 lies outside the unit circle: |z| > 1.
Equivalently, the \psi-weights of the MA(\infty) representation decay geometrically and are absolutely summable.

Invertibility: recover the shocks from the data

The mirror-image condition acts on the MA side. A model is invertible if the unobservable shock \varepsilon_t can be reconstructed as a convergent weighted sum of present and past observed values:

\varepsilon_t = \sum_{j=0}^{\infty} \pi_j\,x_{t-j}, \qquad \sum_{j=0}^{\infty} |\pi_j| < \infty,

which is the AR(\infty) form obtained by "dividing" the other way, \varepsilon_t = \big(\phi(B)/\theta(B)\big)x_t. This matters enormously in practice: forecasting requires estimating the current shock from the data you actually have, and only an invertible model lets you do that stably.

An ARMA process is invertible if and only if every root z of the MA polynomial \theta(z) = 1 + \theta_1 z + \dots + \theta_q z^{q} = 0 lies outside the unit circle: |z| > 1.
Then the \pi-weights of the AR(\infty) representation are absolutely summable.

The picture: roots and the unit circle

Everything lives on one diagram. Plot the polynomial's roots in the complex plane and draw the unit circle |z| = 1. Roots outside the circle (the shaded region beyond it) mean "well-behaved"; a root on or inside the circle breaks the condition. The highlighted points below are safely outside — a real root and a complex-conjugate pair (as an AR(2) can have) — while the labelled point has strayed inside.

For an AR(1), \phi(z) = 1 - \phi z has the single root z = 1/\phi. Demanding |z| > 1 is exactly |\phi| < 1 — the familiar AR(1) stationarity rule, now revealed as the smallest case of the unit-circle law.

Why insist on both?

Causality is about physical sense; invertibility is about identifiability. Here is the key phenomenon. Consider two MA(1) models, x_t = \varepsilon_t + \theta\,\varepsilon_{t-1} and x_t = \varepsilon_t + (1/\theta)\,\varepsilon_{t-1} (with the driving variance adjusted). They have exactly the same autocorrelation function, because

\rho_1 = \frac{\theta}{1+\theta^{2}} = \frac{1/\theta}{1 + (1/\theta)^{2}}.

The ACF cannot distinguish \theta from 1/\theta — and only one of them (the one with |\theta| < 1) is invertible. By agreeing always to choose the invertible representation, the field pins down a single canonical model for each ACF, so parameter estimates are unique and comparable. Requiring both causality and invertibility makes the map from "process" to "parameters" one-to-one.

Worked example

Is x_t = \varepsilon_t + 2\,\varepsilon_{t-1} invertible? The MA polynomial is \theta(z) = 1 + 2z, with root z = -\tfrac12. Since |z| = 0.5 < 1, the root is inside the circle — not invertible. But its twin x_t = \varepsilon_t + \tfrac12\varepsilon_{t-1} has root z = -2 (outside), is invertible, and carries the identical ACF (\rho_1 = 0.4 for both). So the invertible model with \theta = 0.5 is the one we report — same data-fit, unique choice.

Even with both roots outside the circle, a model can still be ill-posed if \phi(B) and \theta(B) share a common factor. If both contain, say, the factor (1 - 0.5B), it cancels from both sides of \phi(B)x_t = \theta(B)\varepsilon_t, leaving a lower-order model. The apparent ARMA(1,1) (1-0.5B)x_t = (1-0.5B)\varepsilon_t is really just white noise. On real data this shows up as a fit where one AR root and one MA root sit almost on top of each other, and the estimator becomes numerically unstable. Always insist that \phi(z) and \theta(z) be coprime — no shared roots — on top of the unit-circle conditions.

A root sitting on the unit circle (|z| = 1) is the knife-edge. On the AR side, an AR(1) with \phi = 1 has its root at z = 1 — this is a unit root, and the process is a non-stationary random walk rather than a causal ARMA. Deciding whether a real series has a root on the circle (needs differencing) or merely near it (a strongly persistent stationary series) is one of the most consequential tests in the subject — the job of the Dickey–Fuller unit-root test. The boundary is where "stationary" ends and "integrated" begins.