Modes of Convergence

For ordinary numbers, "x_n \to x" means one thing. For a sequence of random variables (X_n) there are several genuinely different senses in which it can converge to a limit X — and they are not equivalent. Pinning down which mode you mean is essential, because the limits that define the Itô integral, the law of large numbers, and the central limit theorem each use a different one.

The four modes

Almost surely (X_n \xrightarrow{a.s.} X): the paths converge for every outcome except a set of probability zero — \mathbb{P}\!\left(\lim_{n\to\infty} X_n = X\right) = 1.
In probability (X_n \xrightarrow{\mathbb{P}} X): large gaps become rare — \mathbb{P}\!\left(|X_n - X| > \varepsilon\right) \to 0 \quad\text{for every } \varepsilon > 0.
In L^p (mean of order p; p = 2 is mean-square): the average error shrinks — \mathbb{E}\!\left[|X_n - X|^p\right] \to 0.
In distribution (X_n \xrightarrow{d} X): only the shape converges — F_{X_n}(x) \to F_X(x) at every point x where F_X is continuous. (The continuity proviso matters: at a jump of F_X the CDFs need not agree.)

The implication hierarchy

The modes line up in a strict order. Step through the diagram: both almost-sure and L^p convergence imply convergence in probability, which in turn implies convergence in distribution. None of the arrows reverse, and there is no arrow between a.s. and L^p in either direction.

X_n \xrightarrow{a.s.} X \;\Longrightarrow\; X_n \xrightarrow{\mathbb{P}} X \;\Longrightarrow\; X_n \xrightarrow{d} X, \qquad X_n \xrightarrow{L^p} X \;\Longrightarrow\; X_n \xrightarrow{\mathbb{P}} X.

For random variables on a common probability space, X_n \xrightarrow{a.s.} X \;\Longrightarrow\; X_n \xrightarrow{\mathbb{P}} X \;\Longrightarrow\; X_n \xrightarrow{d} X, and, for any p \ge 1, X_n \xrightarrow{L^p} X \;\Longrightarrow\; X_n \xrightarrow{\mathbb{P}} X. None of these implications reverse in general, and there is no implication between almost-sure and L^p convergence in either direction.

Proof that L^p ⇒ in probability

Of the arrows above, the cleanest to prove is L^p \Rightarrow \mathbb{P}, and it rests on a single workhorse inequality. We first establish Markov's inequality, then read Chebyshev off it, then deploy it on |X_n - X|.

Markov's inequality. Let Y \ge 0 be a non-negative random variable and a > 0 a threshold. The claim is \mathbb{P}(Y \ge a) \le \mathbb{E}[Y]/a. The trick is to bound Y below by a two-step "staircase". Introduce the indicator \mathbb{1}_{\{Y \ge a\}}, which is 1 when Y \ge a and 0 otherwise. Then pointwise (check both cases — if Y \ge a the right side is a \le Y; if Y < a the right side is 0 \le Y):

Y \;\ge\; Y\cdot \mathbb{1}_{\{Y \ge a\}} \;\ge\; a\cdot \mathbb{1}_{\{Y \ge a\}}.

Take expectations across the chain; expectation preserves \ge and is linear, and \mathbb{E}[\mathbb{1}_{\{Y \ge a\}}] = \mathbb{P}(Y \ge a) by definition of an indicator's expectation:

\mathbb{E}[Y] \;\ge\; \mathbb{E}\!\left[a\,\mathbb{1}_{\{Y \ge a\}}\right] = a\,\mathbb{E}\!\left[\mathbb{1}_{\{Y \ge a\}}\right] = a\,\mathbb{P}(Y \ge a).

Divide both sides by a > 0 (which keeps the inequality direction):

\mathbb{P}(Y \ge a) \;\le\; \frac{\mathbb{E}[Y]}{a}.

That is Markov's inequality. Chebyshev's inequality is the special case p = 2: apply Markov to the non-negative variable Y = |Z - \mathbb{E}Z|^2 with threshold a = \varepsilon^2 to get \mathbb{P}(|Z - \mathbb{E}Z| \ge \varepsilon) \le \operatorname{Var}(Z)/\varepsilon^2 — we will use the same idea now with a general power p.

Now the implication. Suppose X_n \xrightarrow{L^p} X, i.e. \mathbb{E}[|X_n - X|^p] \to 0. Fix \varepsilon > 0; we must show \mathbb{P}(|X_n - X| > \varepsilon) \to 0. Because t \mapsto t^p is strictly increasing on [0,\infty), the event |X_n - X| > \varepsilon is the same event as |X_n - X|^p > \varepsilon^p — raising both sides to the p preserves the inequality:

\mathbb{P}\!\left(|X_n - X| > \varepsilon\right) = \mathbb{P}\!\left(|X_n - X|^p > \varepsilon^p\right).

Apply Markov's inequality to the non-negative variable Y = |X_n - X|^p with threshold a = \varepsilon^p:

\mathbb{P}\!\left(|X_n - X|^p > \varepsilon^p\right) \;\le\; \frac{\mathbb{E}\!\left[|X_n - X|^p\right]}{\varepsilon^p}.

Finally let n \to \infty. The numerator \mathbb{E}[|X_n - X|^p] \to 0 by hypothesis, while \varepsilon^p is a fixed positive constant, so the whole bound tends to 0:

0 \;\le\; \mathbb{P}\!\left(|X_n - X| > \varepsilon\right) \;\le\; \frac{\mathbb{E}\!\left[|X_n - X|^p\right]}{\varepsilon^p} \;\xrightarrow{\,n\to\infty\,}\; 0.

Squeezed between 0 and something going to 0, the probability goes to 0 for every \varepsilon — which is exactly convergence in probability. ∎

Why doesn't L^p (or in-probability) convergence force almost-sure convergence? Here is a concrete counterexample — the typewriter or sliding-block sequence — that converges in probability (and in every L^p) yet at no outcome does the sequence of values settle down.

Work on \Omega = [0,1] with the uniform probability (length). Sweep a window of shrinking width across the interval, restarting each time it reaches the right end. Concretely, group the indices into blocks: block k has k windows, each of width 1/k, namely

\left[0, \tfrac1k\right),\ \left[\tfrac1k, \tfrac2k\right),\ \dots,\ \left[\tfrac{k-1}{k}, 1\right).

Let X_n be the indicator of the n-th such window as n runs through all windows of all blocks in order. The window's width shrinks as 1/k \to 0, so:

It converges in probability to 0. For small \varepsilon, \mathbb{P}(|X_n| > \varepsilon) = \mathbb{P}(X_n = 1) equals the width 1/k of the current window, which \to 0. Likewise \mathbb{E}[|X_n|^p] = 1/k \to 0, so it converges in every L^p too.
It does NOT converge almost surely. Fix any point \omega \in [0,1]. Within every block k, exactly one window covers \omega, so X_n(\omega) = 1 infinitely often; but \omega is missed by all the other windows, so X_n(\omega) = 0 infinitely often as well. The sequence X_n(\omega) therefore oscillates forever between 0 and 1 and has no limit — at every outcome, not just a null set.

The block slides across the interval like a typewriter carriage returning each line: the mass shrinks to nothing (giving convergence in probability and in L^p), but the location keeps revisiting every point (destroying almost-sure convergence). This is exactly why the L^p \Rightarrow \mathbb{P} arrow does not upgrade to L^p \Rightarrow a.s.

Why this matters here

The Itô integral is built as an L^2 (mean-square) limit of integrals of simple processes — mean-square convergence is exactly the notion under which that construction closes up, courtesy of the Itô isometry. Meanwhile the weak law of large numbers is a statement of convergence in probability, the strong law is almost-sure convergence, and the central limit theorem is convergence in distribution. Knowing which mode is in play tells you exactly what each theorem does, and does not, promise.