We have a state-space model: a hidden state \alpha_t that evolves, seen only through noisy observations y_t. Now the central question: given the observations up to now, what is our best estimate of the hidden state — and how sure are we? And having answered it, how do we forecast the next observation, and how do we score how well the model fits so we can estimate its parameters?

Extraordinarily, all of this is delivered by a single, cheap, recursive procedure that never looks back over the whole history — it carries a running summary and updates it one observation at a time. That procedure is the Kalman filter. In a time-series course it is the workhorse behind three distinct jobs — filtering, smoothing and forecasting — and, crucially, it is the machine that computes the likelihood we maximise to fit ARMA and structural models. (For the same filter viewed as an optimal controller's state estimator, see the companion Kalman filter in control theory.)

A Gaussian belief, carried forward

Because the model is linear and every noise is Gaussian, the filter's belief about the state stays Gaussian for all time. So the entire belief is just two numbers (or in the vector case, a mean vector and a covariance matrix):

• a mean a_t — the best estimate of the state;
• a variance P_t — our quantified uncertainty about it.

The filter marches through the data running a two-beat cycle at every time step: predict (project the belief forward through the dynamics) then update (pull it back toward the new observation). This is nothing more than Bayes' theorem applied over and over — recursive Bayesian updating for a linear-Gaussian model — where the prior at each step is last step's posterior pushed forward one tick.

The predict–update equations (scalar local level)

Let us write the recursion for the T = 1, Z = 1 local-level model — state variance Q = \sigma_\eta^2, measurement variance H = \sigma_\varepsilon^2. Write a_{t-1}, P_{t-1} for the filtered estimate after seeing y_{t-1}.

Predict — roll the state forward, and let uncertainty grow by the process noise:

Update — a new observation y_t arrives. Form the innovation (the one-step prediction error) and its variance:

Then correct the estimate by a fraction of that surprise — the fraction being the Kalman gain K_t — and shrink the variance:

The gain K_t = P_{t\mid t-1}/(P_{t\mid t-1} + \sigma_\varepsilon^2) is a number between 0 and 1 that weighs confidence against confidence: a large state uncertainty or a small measurement noise pushes it toward 1 (trust the data, correct hard); a small state uncertainty or a noisy sensor pushes it toward 0 (trust the model, barely budge). The correction a_t = (1-K_t)\,a_{t\mid t-1} + K_t\, y_t is simply a weighted average of where the model thought it was and what the sensor just said.

Worked example: one predict–update cycle

Make it concrete. Local level with process variance \sigma_\eta^2 = 1 and measurement variance \sigma_\varepsilon^2 = 4. Suppose after yesterday we hold a_{t-1} = 10 with P_{t-1} = 3, and today's observation lands at y_t = 16.

• Predict: a_{t\mid t-1} = 10 and P_{t\mid t-1} = 3 + 1 = 4.
• Innovation: v_t = 16 - 10 = 6, with variance F_t = 4 + 4 = 8.
• Gain: K_t = 4/8 = 0.5.
• Update the mean: a_t = 10 + 0.5 \times 6 = 13.
• Update the variance: P_t = (1 - 0.5)\times 4 = 2.

Notice the estimate moved from 10 halfway toward the observation 16, landing at 13, and the uncertainty fell from a predicted 4 down to 2 — the observation taught us something. That is the whole filter in one breath, repeated for every time step.

Watching it lock on

Below, the true hidden state is the smooth line; we see it only through the scattered dots. The filtered estimate is the second line, and the shaded band is its \pm 2\sqrt{P_t} uncertainty. Watch the band at the left: the filter starts unsure (a wide band from a diffuse prior) and, observation by observation, the band narrows and the estimate settles onto the truth. After a few steps the gain and variance reach a steady state and the filter simply tracks.

This shrinking band is the visual signature of Bayesian learning: every observation removes a little uncertainty, until the incoming noise and the wandering state balance and the uncertainty holds steady.

The prize: the likelihood, for free

Here is why the filter is the beating heart of estimation. As it runs, it emits at every step an innovation v_t and its variance F_t. Under the model, these one-step prediction errors are independent and Gaussian, v_t \sim N(0, F_t). That independence is a gift: it lets us write the joint density of the whole series as a simple product, term by term — the prediction-error decomposition.

So the Kalman filter is not only an estimator of the hidden state — it is the engine that turns an intractable-looking joint density into a running sum you can hand to an optimiser. Every time you fit an ARIMA model, a filter like this is quietly running under the hood, decomposing the likelihood into these one-step surprises.

Filtering, smoothing, forecasting

The same recursion answers three different questions, distinguished only by which observations you condition on:

• Filtering — estimate \alpha_t using data up to and including time t: E[\alpha_t \mid y_{1:t}]. This is the real-time estimate, exactly what the forward pass computes.
• Smoothing — estimate \alpha_t using the whole sample y_{1:T}, including future observations: E[\alpha_t \mid y_{1:T}]. A backward pass after the filter sharpens every past estimate using hindsight; smoothed estimates are always at least as precise as filtered ones. This is what you want for describing history (e.g. estimating a past trend).
• Forecasting — predict a future observation y_{T+h} from y_{1:T}: just run the predict step forward h times with no updates. The uncertainty band fans out as it goes, exactly as forecast intervals should.

Three tasks — one algorithm, differing only in the direction you point it. This is the same trio of goals (forecasting among them) that motivated the whole subject, now delivered by a single recursion.