You now have two engines with complementary temperaments. An autoregressive model gives a series long, fading persistence — it leans on its own past and forgets slowly. A moving-average model gives it short, sharp shock memory — a surprise echoes for a few steps, then is gone. Real data usually have both: a smooth underlying momentum plus brief reactions to news. The obvious move is to bolt the two engines together, and that is the ARMA model.

An ARMA(p, q) puts an AR polynomial on the series and an MA polynomial on the noise:

In backshift form the two natures sit on opposite sides of one clean equation:

with \phi(B) = 1 - \phi_1 B - \dots - \phi_p B^{p} and \theta(B) = 1 + \theta_1 B + \dots + \theta_q B^{q}. AR is the special case q = 0; MA is the special case p = 0.

Parsimony: the real reason ARMA exists

You might ask: if I can already build any weight sequence with a long enough AR or a long enough MA, why bother mixing them? The answer is parsimony — fewer parameters for the same fit.

A pure MA representation of a persistent series needs infinitely many terms (MA(\infty)); truncating it to a usable length still costs many coefficients. A pure AR often needs a high order p to bend its ACF into the right shape. But a mixed ARMA(1,1) — just two parameters — can reproduce a decay pattern that would take an AR(1) an awkward number of extra lags, or an MA a whole tail, to match. Because \theta(B)/\phi(B) is a ratio of polynomials, it manufactures rich infinite weight sequences from a tiny number of coefficients — the same efficiency that a rational function has over a Taylor polynomial. Fewer parameters means less to estimate, tighter forecasts, and less overfitting.

The identification headache

Mixing the two engines has a price, and it is paid at the diagnosis stage. Recall the clean fingerprints:

There is the rub: for a genuine ARMA, both the ACF and the PACF tail off — neither cuts cleanly — so neither plot hands you the order on a plate. You lose the "count the spikes" trick that made pure AR and MA so easy to read. Choosing p and q for a mixed model becomes an art of trial, comparison and information criteria — the whole business of identifying ARMA models.

ARMA(1,1), seen

Below is the ACF of an ARMA(1,1) with \phi = 0.6 and \theta = 0.5. After an initial value \rho_1 shaped by the MA term, the correlations settle into a pure geometric decay, \rho_k = \phi\,\rho_{k-1} for k \ge 2 — tailing off toward zero without ever cutting off. That "shrinking but never zero" tail is precisely why you cannot read the order from the picture alone.

Worked example: ARMA(1,1) in backshift form

Start from the raw recursion

Collect the x terms on the left and the \varepsilon terms on the right, replacing each lag by a power of B:

So \phi(B) = 1 - 0.6B and \theta(B) = 1 + 0.5B. Dividing gives the MA(\infty) weights, \psi(B) = (1+0.5B)/(1-0.6B); expanding the denominator as a geometric series yields \psi_0 = 1 and, for j \ge 1, \psi_j = (0.6 + 0.5)\,0.6^{\,j-1} = 1.1\times 0.6^{\,j-1} — two numbers generating an infinite, smoothly decaying tail.