Probability Spaces

Suppose your desk asks you to price an option — a contract that pays out only if a stock finishes above a strike. Its value today clearly depends on what the stock does tomorrow, and what the stock does tomorrow is random. So before you can write down a single pricing formula, you must answer a question that sounds almost philosophical: what, precisely, does "random" mean?

Mathematics answers with a contract of its own. To speak of chance at all, you must put three things on the table:

Formally, a probability measure is nothing more than a measure with one extra requirement: the whole space has measure one. Write it \mathbb{P}, so that \mathbb{P}(\Omega) = 1something is certain to happen. The complete object,

(\Omega, \mathcal{F}, \mathbb{P}),

is a probability space: \Omega the sample space of outcomes, \mathcal{F} the σ-algebra of events, and \mathbb{P} the probability assigned to each event. This triple is the stage on which every random phenomenon in this course plays out — every stock path, every default, every roll of the die is a point \omega moving through this machinery. Learn to build the triple explicitly for small examples now, and the continuous-time models later will feel like the same construction with a bigger \Omega.

Kolmogorov's axioms

Stripped to essentials, a probability measure obeys three axioms — written down by Andrey Kolmogorov in 1933, they are the foundation on which the rest of probability is built entirely:

Notice what is not on the list: no mention of fairness, symmetry, frequencies, or beliefs. The axioms are deliberately agnostic about what probability "really is" — they only pin down how it must behave. That neutrality is a feature: the same three rules govern a casino die, a default probability calibrated from bond spreads, and the artificial "risk-neutral" odds used to price derivatives.

From these alone come the everyday rules. Let us derive the three workhorses line by line, using only the axioms above.

The complement rule. An event A and its complement A^{c} share no outcome and between them cover every outcome, so they form a disjoint union of the whole space:

\Omega = A \sqcup A^{c}.

Take \mathbb{P} of both sides. On the left, normalisation gives \mathbb{P}(\Omega) = 1; on the right, additivity over the two disjoint pieces splits the probability into a sum:

1 = \mathbb{P}(\Omega) = \mathbb{P}(A \sqcup A^{c}) = \mathbb{P}(A) + \mathbb{P}(A^{c}).

Rearranging the equation 1 = \mathbb{P}(A) + \mathbb{P}(A^{c}) isolates the complement:

\mathbb{P}(A^{c}) = 1 - \mathbb{P}(A).

The empty set has probability zero. The empty set is the complement of the whole space, \emptyset = \Omega^{c}, so feed A = \Omega into the rule we just proved:

\mathbb{P}(\emptyset) = \mathbb{P}(\Omega^{c}) = 1 - \mathbb{P}(\Omega) = 1 - 1 = 0.

Inclusion–exclusion. For two events A and B that may overlap, additivity does not apply directly — but it does once we rewrite the pieces as disjoint unions. First split A \cup B into A together with the part of B lying outside A; these are disjoint and cover the union:

A \cup B = A \sqcup (B \setminus A).

Likewise split B itself into the part inside A and the part outside it:

B = (A \cap B) \sqcup (B \setminus A).

Apply additivity to each disjoint split:

\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B \setminus A), \mathbb{P}(B) = \mathbb{P}(A \cap B) + \mathbb{P}(B \setminus A).

The second equation hands us the leftover piece, \mathbb{P}(B \setminus A) = \mathbb{P}(B) - \mathbb{P}(A \cap B). Substitute that into the first:

\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B).

The - \mathbb{P}(A \cap B) is precisely the overlap, which \mathbb{P}(A) + \mathbb{P}(B) would otherwise count twice.

Countable additivity has a second face: probability is continuous along nested limits. Suppose events increase to a limit, A_1 \subseteq A_2 \subseteq A_3 \subseteq \cdots with A = \bigcup_n A_n (written A_n \uparrow A). Disjointify the increasing chain into the fresh ring added at each stage,

B_1 = A_1, \qquad B_n = A_n \setminus A_{n-1},

so the B_n are pairwise disjoint with A_n = B_1 \sqcup \dots \sqcup B_n and A = \bigsqcup_n B_n. Countable additivity on the full disjoint union, then recognising the partial sums, gives

\mathbb{P}(A) = \sum_{n=1}^{\infty} \mathbb{P}(B_n) = \lim_{N \to \infty} \sum_{n=1}^{N} \mathbb{P}(B_n) = \lim_{N \to \infty} \mathbb{P}(A_N).

So A_n \uparrow A \Rightarrow \mathbb{P}(A_n) \uparrow \mathbb{P}(A). The decreasing version, A_n \downarrow A, follows by complementing. This continuity is what lets us compute a probability as the limit of approximations — the bridge from finite events to genuinely infinite ones.

On any probability space (\Omega, \mathcal{F}, \mathbb{P}) and for all events A, B:

All four follow from non-negativity, normalisation, and countable additivity alone.

Building a probability space by hand: one coin toss

The definition earns its keep only when you can construct the triple explicitly. Start with the smallest interesting experiment: toss one fair coin.

Step 1 — list the possible worlds. Two ways it can land:

\Omega = \{H, T\}.

Step 2 — choose the askable questions. For a finite \Omega the natural choice is every subset — the power set:

\mathcal{F} = 2^{\Omega} = \bigl\{\, \emptyset,\; \{H\},\; \{T\},\; \Omega \,\bigr\}.

Read each event as a question: \emptyset asks "did nothing happen?" (never yes), \{H\} asks "heads?", \{T\} asks "tails?", and \Omega asks "did the coin land at all?" (always yes). Check the σ-algebra requirements by eye: it contains \Omega, it is closed under complements (\{H\}^{c} = \{T\}, \Omega^{c} = \emptyset), and closed under unions (\{H\} \cup \{T\} = \Omega). ✓

Step 3 — assign the odds. Fairness means

\mathbb{P}(\emptyset) = 0, \qquad \mathbb{P}(\{H\}) = \mathbb{P}(\{T\}) = \tfrac{1}{2}, \qquad \mathbb{P}(\Omega) = 1.

Step 4 — verify the axioms. Non-negativity: all four values are \ge 0. ✓ Normalisation: \mathbb{P}(\Omega) = 1. ✓ Additivity: the only non-trivial disjoint union is \{H\} \sqcup \{T\} = \Omega, and indeed \tfrac{1}{2} + \tfrac{1}{2} = 1 = \mathbb{P}(\Omega). ✓ All three axioms hold, so (\Omega, \mathcal{F}, \mathbb{P}) is a genuine probability space — the smallest one worth having.

Note the division of labour: nothing about \Omega or \mathcal{F} knew the coin was fair. A biased coin lives on the same stage (\Omega, \mathcal{F}) with a different \mathbb{P} — say \mathbb{P}(\{H\}) = 0.6, \mathbb{P}(\{T\}) = 0.4. Same worlds, same questions, different odds. Keep that separation in mind; finance leans on it hard.

Two tosses: the space grows fast

Now toss the fair coin twice, recording the ordered pair of results. The possible worlds multiply:

\Omega = \{HH,\; HT,\; TH,\; TT\}, \qquad |\Omega| = 2^2 = 4.

Take \mathcal{F} = 2^{\Omega} again — with four outcomes the power set already holds 2^4 = 16 events (a coin tossed n times has 2^n outcomes and a power set of 2^{2^n} events; by n = 5 that is over four billion askable questions). Fairness and independence make every outcome equally likely, so the measure is uniform counting:

\mathbb{P}(A) = \frac{|A|}{4} \quad \text{for every } A \in \mathcal{F}.

The axioms are quick to verify: |A| \ge 0 gives non-negativity; \mathbb{P}(\Omega) = 4/4 = 1 gives normalisation; and disjoint events have |A \sqcup B| = |A| + |B|, so additivity is just counting.

A compound event, two ways. What is the probability of at least one head? Call that event C.

Via the complement rule. The complement of "at least one head" is "no heads at all", the single world C^{c} = \{TT\}:

\mathbb{P}(C) = 1 - \mathbb{P}(\{TT\}) = 1 - \tfrac{1}{4} = \tfrac{3}{4}.

Via inclusion–exclusion. Write C = A \cup B where A = \{HH, HT\} ("first toss heads") and B = \{HH, TH\} ("second toss heads"). They overlap on A \cap B = \{HH\}, so

\mathbb{P}(A \cup B) = \tfrac{2}{4} + \tfrac{2}{4} - \tfrac{1}{4} = \tfrac{3}{4}.

Two independent routes through the axioms, one answer — exactly the sort of consistency the three rules guarantee. When a probability computation ever disagrees with itself, an axiom has been violated somewhere, and the axioms tell you where to look.

The one-step binomial stock: finance's favourite probability space

Here is the probability space that launches every derivatives course. A stock is worth S_0 today; by tomorrow it will have either risen or fallen. The possible worlds could not be simpler:

\Omega = \{\text{up},\; \text{down}\}, \qquad \mathcal{F} = \bigl\{\, \emptyset,\; \{\text{up}\},\; \{\text{down}\},\; \Omega \,\bigr\}.

In the world \omega = \text{up} the stock moves to uS_0; in \omega = \text{down}, to dS_0 (with d < 1 < u). The measure needs only one number, the up-probability p:

\mathbb{P}(\{\text{up}\}) = p, \qquad \mathbb{P}(\{\text{down}\}) = 1 - p, \qquad 0 < p < 1.

Normalisation and additivity hold by construction: p + (1 - p) = 1 = \mathbb{P}(\Omega). Structurally this is the coin-toss space wearing a suit — which is precisely the point. A call option struck between dS_0 and uS_0 pays out exactly on the event \{\text{up}\} \in \mathcal{F}, so "what is the option worth?" is an askable question the moment the triple is written down.

And now the payoff of separating stage from odds: the deepest trick in derivative pricing is to keep (\Omega, \mathcal{F}) fixed but swap the measure — replace the real-world \mathbb{P} with an artificial risk-neutral measure \mathbb{Q} under which discounted prices behave like fair bets. That manoeuvre only makes sense because the triple keeps the worlds, the questions, and the odds as three separate dials. Chain many of these one-step spaces together and you are most of the way to the binomial tree, and from there to Black–Scholes.

Three misconceptions trip up nearly everyone meeting the triple for the first time:

Pascal and Fermat traded letters about gambling problems in 1654, and for the next three centuries probability produced brilliant results — Bernoulli's law of large numbers, de Moivre's bell curve, Laplace's celestial mechanics of chance — without anyone being able to say precisely what a probability was. Definitions in terms of "equally likely cases" were circular (equally likely?), and frequency definitions wobbled on what counts as a limit of experiments.

The cracks became scandalous with Bertrand's paradox (1889): asked for the probability that a random chord of a circle is longer than the side of the inscribed equilateral triangle, three perfectly reasonable methods give three different answers — \tfrac{1}{2}, \tfrac{1}{3}, and \tfrac{1}{4}. The paradox dissolves once you insist the question is ill-posed until someone exhibits the triple: "random chord" means nothing until \Omega, \mathcal{F} and \mathbb{P} are chosen, and the three methods silently chose three different measures. Hilbert listed the axiomatisation of probability in his famous 1900 problems.

The answer arrived in 1933, when Andrey Kolmogorov's slim book Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability") made the then-radical identification: probability is measure theory with total mass one. Sixty pages, and three hundred years of unease evaporated — random variables became measurable functions, expectation became an integral, and the road opened to conditional expectation, martingales, and eventually the stochastic calculus that modern finance runs on. Not bad for a book shorter than most option-pricing manuals.

The fair die, end to end

One more finite space, this time with room to explore. Take \Omega = \{1,2,3,4,5,6\}, \mathcal{F} = 2^{\Omega} the power set (all 2^6 = 64 events), and \mathbb{P}(A) = |A| / 6 — each outcome equally likely. Pick an event below; its faces light up and its probability shows as a fraction. Every choice obeys 0 \le \mathbb{P}(A) \le 1.

Let us actually compute with this space, checking each rule against the formula \mathbb{P}(A) = |A|/6.

\mathbb{P}(A \cup B) = \tfrac{3}{6} + \tfrac{3}{6} - \tfrac{2}{6} = \tfrac{4}{6} = \tfrac{2}{3},

and counting directly confirms it: A \cup B = \{2, 4, 5, 6\} has four outcomes, so \mathbb{P}(A \cup B) = \tfrac{4}{6} = \tfrac{2}{3}. The -\,\tfrac{2}{6} removed exactly the double-counted overlap \{4, 6\}.

As you switch events in the figure, watch how the machinery never changes — the six worlds and the sixty-four questions are fixed once and for all; only which question you are asking moves. That is the probability-space discipline in miniature.