You have exactly one history of the world. The FTSE closed where it closed yesterday; the Thames rose to the level it rose to; your own heart beat the beats it beat. When we sit down to model a time series, that single recorded history is all we will ever get to see — and yet the whole enterprise rests on imagining the histories that could have happened but didn't.

That imagined family of possible histories is a stochastic process. The one we actually observed is a single realization — one draw, one sample path, plucked out of an invisible ensemble. Getting this distinction clear is the most important conceptual step in the entire subject, because almost every quantity we estimate is a property of the process, while every number we measure comes from the one realization.

The formal definition

A stochastic process is nothing more exotic than a family of random variables indexed by time:

one random variable X_t for every time index t in the index set \mathcal{T} (for us, usually the integers t = 1, 2, \dots). Each X_t lives on the same underlying probability space, so it has its own distribution — but crucially the variables at different times are joined by a joint distribution that encodes how they move together. That joint structure is the serial dependence.

A useful mental model is a two-dimensional table with an axis you can never fully explore:

Reading down a column is the ensemble view; reading along a row is the time view. Your data is exactly one row. Everything hard about time series comes from wanting column-facts from a single row.

One process, many paths

The figure shows four sample paths generated by the same stochastic process. They share a family resemblance — the same tendency to drift, the same roughness — yet no two agree at any single time. Nature hands you exactly one of these curves and hides the rest. The dashed horizontal line is the process mean function \mu_t: the average across all the paths at each time, the thing you would see if you could read down the ensemble column.

Squint at any one coloured path and you would struggle to guess the flat mean line — a single wandering realization looks nothing like the average of the ensemble. That gap between "what one path does" and "what the process is" is precisely the difficulty that ergodicity will later rescue us from.

Describing a process: moments and finite-dimensional laws

Since we can rarely write down the full joint law of infinitely many variables, we describe a process through a hierarchy. The complete description is its collection of finite-dimensional distributions — the joint law of every finite selection (X_{t_1}, X_{t_2}, \dots, X_{t_n}). Knowing all of these, for all choices of times, pins the process down completely (Kolmogorov's theorem guarantees such a process exists).

In practice we usually settle for the first two moment functions, which are all a linear-Gaussian world needs:

The mean function \mu_t tracks the level at each time; the autocovariance function \gamma(s,t) measures how the value at time s co-varies with the value at time t — dependence spread across the whole grid of time pairs. Both are functions of the process, defined by averaging down the ensemble; both are what our single realization must somehow let us estimate.

A worked example: the shifting-level process

Let X_t = A + \varepsilon_t, where A \sim \mathcal{N}(0, \tau^2) is drawn once at the start of time and then frozen, and each \varepsilon_t \sim \mathcal{N}(0, \sigma^2) is a fresh independent shock. Then for any times s \neq t:

• \mu_t = \mathbb{E}[A] + \mathbb{E}[\varepsilon_t] = 0 for every t — a flat mean function.
• \gamma(t,t) = \operatorname{Var}(A) + \operatorname{Var}(\varepsilon_t) = \tau^2 + \sigma^2.
• \gamma(s,t) = \operatorname{Cov}(A + \varepsilon_s,\ A + \varepsilon_t) = \operatorname{Var}(A) = \tau^2, because the shared A is the only thing the two times have in common.

Notice: a single realization of this process is a jittery series sitting at a level A that never moves — but a different realization sits at a different level. Averaging one path over time tells you about that path's own A, not about the ensemble mean 0. Hold on to this example: it is the classic warning that a single path can systematically mislead you, and we meet it again as the standard non-ergodic counterexample.