Identifying ARMA Models

A fresh, stationary series lands on your desk. You believe some ARMA model generated it — but which? How many autoregressive lags p, how many moving-average lags q? This is the identification step, the first and most judgement-laden stage of the whole Box–Jenkins method. The core tool is wonderfully visual: plot the two correlograms — the ACF and the PACF — side by side, and read their shapes together.

The logic rests on the two signatures you already know. An MA cuts off in the ACF; an AR cuts off in the PACF. Put them in one table and identification becomes pattern-matching.

The identification table

AR(p): ACF tails off; PACF cuts off after lag p. → read p from the PACF.
MA(q): ACF cuts off after lag q; PACF tails off. → read q from the ACF.
ARMA(p,q): both tail off — neither cuts off, so no clean spike count; start small and compare.

Model	ACF behaviour	PACF behaviour	What to read
AR(p)	tails off (geometric / damped sine)	cuts off after lag p	p = last PACF spike
MA(q)	cuts off after lag q	tails off	q = last ACF spike
ARMA(p,q)	tails off	tails off	try small (p,q); use a criterion

Memorise the diagonal: the plot that cuts off names its order, and the other one tails. If both tail off, you are in mixed ARMA territory and the plots alone will not hand you the answer.

A worked pair of correlograms

Here is the diagnostic in action for an AR(1) with \phi = 0.7. The ACF (left idea) decays geometrically, \rho_k = 0.7^{k}, dying away but never truly hitting zero — it tails off. The PACF shows a single decisive spike at lag 1 and is then flat — it cuts off at lag 1. Reading the pair by the table: "ACF tails, PACF cuts at 1" ⟹ AR(1). That is the whole method in one example.

Swap the two pictures — an ACF that cuts off at lag 1 while the PACF tails — and you would instead read MA(1). The shapes, not the individual numbers, carry the message.

When the plots won't decide: criteria

For a genuine mixed ARMA both correlograms tail off, and eyeballing cannot separate, say, ARMA(1,1) from ARMA(2,1). Two remedies:

The extended ACF (EACF) lays out a grid over candidate (p,q) pairs and marks a triangle of zeros whose corner points at a sensible order — a purpose-built tool for the mixed case.
Information criteria — AIC, the small-sample-corrected AICc, and BIC — score each fitted candidate by trading goodness-of-fit against the number of parameters. You fit a handful of small models and pick the lowest score. This model-selection step turns identification from art into a semi-automatic search (modern "auto-ARIMA" routines do exactly this).

In practice you combine them: let the ACF/PACF narrow the field to a few plausible orders, then let a criterion (and a residual check) pick the winner.

The clean "tails vs cuts" pictures are population truths. Your sample plots are estimates from finite data and jiggle around those ideals, with confidence bands at about \pm 1.96/\sqrt{T}. A bar poking just past the band at lag 7 is usually noise, not a real order-7 term. Two disciplines keep you honest: (1) parsimony — prefer the smallest order consistent with the plots, since every extra parameter is another thing to estimate badly; and (2) diagnostic checking — after fitting, the residuals should look like white noise (a flat residual ACF, a passing portmanteau test). Identification is a hypothesis, confirmed only when the residuals come out clean. Never read a single noisy spike as gospel.

It is tempting to throw ARMA(5,5) at everything — surely a bigger model fits better? It does fit the training data better, always. But over-parameterised models chase noise, invite near-common-factor redundancy (unstable, near-cancelling roots), and forecast worse out of sample — their extra coefficients add variance without adding signal. Parsimony is not mere tidiness; it is the principle that a model should be as simple as the data allow. The whole ACF/PACF ritual exists to find the smallest order that captures the dependence, and the information criteria formalise that instinct by charging a penalty for every parameter.