Probability Spaces
Suppose your desk asks you to price an option — a contract that pays out only if a stock
finishes above a strike. Its value today clearly depends on what the stock does tomorrow,
and what the stock does tomorrow is random. So before you can write down a single
pricing formula, you must answer a question that sounds almost philosophical:
what, precisely, does "random" mean?
Mathematics answers with a contract of its own. To speak of chance at all, you must put
three things on the table:
-
the possible worlds — a set \Omega listing
every way the experiment could turn out, one outcome
\omega per world;
-
the askable questions — a collection \mathcal{F}
of subsets of \Omega, the events: "did the stock go
up?", "did at least one coin land heads?" — each question is the set of worlds
in which its answer is yes;
-
the odds — a function \mathbb{P} attaching a
number in [0,1] to every askable question.
Formally, a probability measure is nothing more than a
measure
with one extra requirement: the whole space has measure one. Write it
\mathbb{P}, so that
\mathbb{P}(\Omega) = 1 — something is certain to
happen. The complete object,
(\Omega, \mathcal{F}, \mathbb{P}),
is a probability space:
\Omega the sample space of outcomes,
\mathcal{F} the σ-algebra of events, and
\mathbb{P} the probability assigned to each event. This
triple is the stage on which every random phenomenon in this course plays out — every stock
path, every default, every roll of the die is a point \omega
moving through this machinery. Learn to build the triple explicitly for small examples now,
and the continuous-time models later will feel like the same construction with a bigger
\Omega.
Kolmogorov's axioms
Stripped to essentials, a probability measure obeys three axioms — written down by
Andrey Kolmogorov in 1933, they are the
foundation on which the rest of probability is built entirely:
- Non-negativity: \mathbb{P}(A) \ge 0 for every event A.
- Normalisation: \mathbb{P}(\Omega) = 1.
-
Countable additivity: for pairwise-disjoint
A_1, A_2, \dots,
\mathbb{P}\!\left(\bigcup_{i} A_i\right) = \sum_{i} \mathbb{P}(A_i).
Notice what is not on the list: no mention of fairness, symmetry, frequencies, or
beliefs. The axioms are deliberately agnostic about what probability "really is" — they only
pin down how it must behave. That neutrality is a feature: the same three rules
govern a casino die, a default probability calibrated from bond spreads, and the artificial
"risk-neutral" odds used to price derivatives.
From these alone come the everyday rules. Let us derive the three workhorses line by line,
using only the axioms above.
The complement rule. An event A and its
complement A^{c} share no outcome and between them cover every
outcome, so they form a disjoint union of the whole space:
\Omega = A \sqcup A^{c}.
Take \mathbb{P} of both sides. On the left, normalisation gives
\mathbb{P}(\Omega) = 1; on the right, additivity over the two
disjoint pieces splits the probability into a sum:
1 = \mathbb{P}(\Omega) = \mathbb{P}(A \sqcup A^{c}) = \mathbb{P}(A) + \mathbb{P}(A^{c}).
Rearranging the equation 1 = \mathbb{P}(A) + \mathbb{P}(A^{c})
isolates the complement:
\mathbb{P}(A^{c}) = 1 - \mathbb{P}(A).
The empty set has probability zero. The empty set is the complement of the
whole space, \emptyset = \Omega^{c}, so feed
A = \Omega into the rule we just proved:
\mathbb{P}(\emptyset) = \mathbb{P}(\Omega^{c}) = 1 - \mathbb{P}(\Omega) = 1 - 1 = 0.
Inclusion–exclusion. For two events A and
B that may overlap, additivity does not apply directly — but it
does once we rewrite the pieces as disjoint unions. First split
A \cup B into A together with the part
of B lying outside A; these are
disjoint and cover the union:
A \cup B = A \sqcup (B \setminus A).
Likewise split B itself into the part inside
A and the part outside it:
B = (A \cap B) \sqcup (B \setminus A).
Apply additivity to each disjoint split:
\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B \setminus A),
\mathbb{P}(B) = \mathbb{P}(A \cap B) + \mathbb{P}(B \setminus A).
The second equation hands us the leftover piece,
\mathbb{P}(B \setminus A) = \mathbb{P}(B) - \mathbb{P}(A \cap B).
Substitute that into the first:
\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B).
The - \mathbb{P}(A \cap B) is precisely the overlap, which
\mathbb{P}(A) + \mathbb{P}(B) would otherwise count twice.
Countable additivity has a second face: probability is continuous along nested
limits. Suppose events increase to a limit,
A_1 \subseteq A_2 \subseteq A_3 \subseteq \cdots with
A = \bigcup_n A_n (written
A_n \uparrow A). Disjointify the increasing chain into the
fresh ring added at each stage,
B_1 = A_1, \qquad B_n = A_n \setminus A_{n-1},
so the B_n are pairwise disjoint with
A_n = B_1 \sqcup \dots \sqcup B_n and
A = \bigsqcup_n B_n. Countable additivity on the full disjoint
union, then recognising the partial sums, gives
\mathbb{P}(A) = \sum_{n=1}^{\infty} \mathbb{P}(B_n) = \lim_{N \to \infty} \sum_{n=1}^{N} \mathbb{P}(B_n) = \lim_{N \to \infty} \mathbb{P}(A_N).
So A_n \uparrow A \Rightarrow \mathbb{P}(A_n) \uparrow \mathbb{P}(A).
The decreasing version, A_n \downarrow A, follows by
complementing. This continuity is what lets us compute a probability as the limit of
approximations — the bridge from finite events to genuinely infinite ones.
On any probability space (\Omega, \mathcal{F}, \mathbb{P}) and for
all events A, B:
- \mathbb{P}(A^{c}) = 1 - \mathbb{P}(A).
- \mathbb{P}(\emptyset) = 0.
- 0 \le \mathbb{P}(A) \le 1.
-
The inclusion–exclusion identity
\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B).
All four follow from non-negativity, normalisation, and countable additivity alone.
Building a probability space by hand: one coin toss
The definition earns its keep only when you can construct the triple explicitly. Start with
the smallest interesting experiment: toss one fair coin.
Step 1 — list the possible worlds. Two ways it can land:
\Omega = \{H, T\}.
Step 2 — choose the askable questions. For a finite
\Omega the natural choice is every subset — the power
set:
\mathcal{F} = 2^{\Omega} = \bigl\{\, \emptyset,\; \{H\},\; \{T\},\; \Omega \,\bigr\}.
Read each event as a question: \emptyset asks "did nothing
happen?" (never yes), \{H\} asks "heads?",
\{T\} asks "tails?", and \Omega asks
"did the coin land at all?" (always yes). Check the σ-algebra requirements by eye: it
contains \Omega, it is closed under complements
(\{H\}^{c} = \{T\}, \Omega^{c} = \emptyset),
and closed under unions (\{H\} \cup \{T\} = \Omega). ✓
Step 3 — assign the odds. Fairness means
\mathbb{P}(\emptyset) = 0, \qquad \mathbb{P}(\{H\}) = \mathbb{P}(\{T\}) = \tfrac{1}{2}, \qquad \mathbb{P}(\Omega) = 1.
Step 4 — verify the axioms. Non-negativity: all four values are
\ge 0. ✓ Normalisation:
\mathbb{P}(\Omega) = 1. ✓ Additivity: the only non-trivial
disjoint union is \{H\} \sqcup \{T\} = \Omega, and indeed
\tfrac{1}{2} + \tfrac{1}{2} = 1 = \mathbb{P}(\Omega). ✓ All
three axioms hold, so (\Omega, \mathcal{F}, \mathbb{P}) is a
genuine probability space — the smallest one worth having.
Note the division of labour: nothing about \Omega or
\mathcal{F} knew the coin was fair. A biased coin lives on the
same stage (\Omega, \mathcal{F}) with a different
\mathbb{P} — say
\mathbb{P}(\{H\}) = 0.6,
\mathbb{P}(\{T\}) = 0.4. Same worlds, same questions, different
odds. Keep that separation in mind; finance leans on it hard.
Two tosses: the space grows fast
Now toss the fair coin twice, recording the ordered pair of results. The possible worlds
multiply:
\Omega = \{HH,\; HT,\; TH,\; TT\}, \qquad |\Omega| = 2^2 = 4.
Take \mathcal{F} = 2^{\Omega} again — with four outcomes the
power set already holds 2^4 = 16 events (a coin tossed
n times has 2^n outcomes and a power
set of 2^{2^n} events; by n = 5 that
is over four billion askable questions). Fairness and independence make every outcome
equally likely, so the measure is uniform counting:
\mathbb{P}(A) = \frac{|A|}{4} \quad \text{for every } A \in \mathcal{F}.
The axioms are quick to verify: |A| \ge 0 gives non-negativity;
\mathbb{P}(\Omega) = 4/4 = 1 gives normalisation; and disjoint
events have |A \sqcup B| = |A| + |B|, so additivity is just
counting.
A compound event, two ways. What is the probability of at least one
head? Call that event C.
Via the complement rule. The complement of "at least one head" is "no heads at
all", the single world C^{c} = \{TT\}:
\mathbb{P}(C) = 1 - \mathbb{P}(\{TT\}) = 1 - \tfrac{1}{4} = \tfrac{3}{4}.
Via inclusion–exclusion. Write C = A \cup B where
A = \{HH, HT\} ("first toss heads") and
B = \{HH, TH\} ("second toss heads"). They overlap on
A \cap B = \{HH\}, so
\mathbb{P}(A \cup B) = \tfrac{2}{4} + \tfrac{2}{4} - \tfrac{1}{4} = \tfrac{3}{4}.
Two independent routes through the axioms, one answer — exactly the sort of consistency the
three rules guarantee. When a probability computation ever disagrees with itself,
an axiom has been violated somewhere, and the axioms tell you where to look.
The one-step binomial stock: finance's favourite probability space
Here is the probability space that launches every derivatives course. A stock is worth
S_0 today; by tomorrow it will have either risen or fallen. The
possible worlds could not be simpler:
\Omega = \{\text{up},\; \text{down}\}, \qquad \mathcal{F} = \bigl\{\, \emptyset,\; \{\text{up}\},\; \{\text{down}\},\; \Omega \,\bigr\}.
In the world \omega = \text{up} the stock moves to
uS_0; in \omega = \text{down}, to
dS_0 (with d < 1 < u). The
measure needs only one number, the up-probability p:
\mathbb{P}(\{\text{up}\}) = p, \qquad \mathbb{P}(\{\text{down}\}) = 1 - p, \qquad 0 < p < 1.
Normalisation and additivity hold by construction:
p + (1 - p) = 1 = \mathbb{P}(\Omega). Structurally this is the
coin-toss space wearing a suit — which is precisely the point. A call option struck between
dS_0 and uS_0 pays out exactly on the
event \{\text{up}\} \in \mathcal{F}, so "what is the option
worth?" is an askable question the moment the triple is written down.
And now the payoff of separating stage from odds: the deepest trick in derivative pricing is
to keep (\Omega, \mathcal{F}) fixed but swap the
measure — replace the real-world \mathbb{P} with an
artificial
risk-neutral measure
\mathbb{Q} under which discounted prices behave like fair bets.
That manoeuvre only makes sense because the triple keeps the worlds, the questions, and the
odds as three separate dials. Chain many of these one-step spaces together and you are most
of the way to the binomial tree, and from there to Black–Scholes.
Three misconceptions trip up nearly everyone meeting the triple for the first time:
-
\mathcal{F} is not always the power set. For
our finite examples it was, and that is fine. But for an infinite
\Omega — say, choosing a random point of
[0,1] — it cannot be: there exist genuinely
pathological subsets (the non-measurable sets, built with the axiom of choice) to which no
length-like probability can be consistently assigned at all. The fix is to shrink
ambition: \mathcal{F} keeps only the measurable sets,
the ones probability can actually be defined on. "The askable questions" is exactly the
right intuition — some questions about an infinite space simply cannot be asked.
-
\mathbb{P} assigns probability to events, not
outcomes. The input to \mathbb{P} is a member of
\mathcal{F} — a set. For a single coin,
\mathbb{P}(\{H\}) is legal but
"\mathbb{P}(H)" is, strictly, a type error (we write it as
shorthand, but the braces are doing real work). The distinction bites hard in continuous
models: a stock price has \mathbb{P}(\{\omega\}) = 0 for
every individual world, yet events like "the price ends above the strike" carry
all the probability. Outcome probabilities do not determine the measure there — event
probabilities do.
-
Probability zero is not impossibility. In the continuous setting just
mentioned, some outcome certainly occurs even though each has probability zero.
"Almost surely" — probability one — is the strongest guarantee measure-theoretic
probability ever gives, and it tolerates exceptions on probability-zero sets. Quants say
"a.s." constantly for exactly this reason.
Pascal and Fermat traded letters about gambling problems in 1654, and for the next three
centuries probability produced brilliant results — Bernoulli's law of large numbers,
de Moivre's bell curve, Laplace's celestial mechanics of chance — without anyone being
able to say precisely what a probability was. Definitions in terms of "equally likely
cases" were circular (equally likely?), and frequency definitions wobbled on what
counts as a limit of experiments.
The cracks became scandalous with Bertrand's paradox (1889): asked for the
probability that a random chord of a circle is longer than the side of the inscribed
equilateral triangle, three perfectly reasonable methods give three different answers —
\tfrac{1}{2}, \tfrac{1}{3}, and
\tfrac{1}{4}. The paradox dissolves once you insist the question
is ill-posed until someone exhibits the triple: "random chord" means nothing until
\Omega, \mathcal{F} and
\mathbb{P} are chosen, and the three methods silently chose three
different measures. Hilbert listed the axiomatisation of probability in his famous 1900
problems.
The answer arrived in 1933, when Andrey Kolmogorov's slim book
Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of
Probability") made the then-radical identification: probability is measure theory
with total mass one. Sixty pages, and three hundred years of unease evaporated — random
variables became measurable functions, expectation became an integral, and the road opened
to conditional expectation, martingales, and eventually the stochastic calculus that modern
finance runs on. Not bad for a book shorter than most option-pricing manuals.
The fair die, end to end
One more finite space, this time with room to explore. Take
\Omega = \{1,2,3,4,5,6\},
\mathcal{F} = 2^{\Omega} the power set (all
2^6 = 64 events), and
\mathbb{P}(A) = |A| / 6 — each outcome equally likely. Pick an
event below; its faces light up and its probability shows as a fraction. Every choice obeys
0 \le \mathbb{P}(A) \le 1.
Let us actually compute with this space, checking each rule against the formula
\mathbb{P}(A) = |A|/6.
-
An even face. Let E = \{2, 4, 6\}. It has
three outcomes, so
\mathbb{P}(E) = \tfrac{|E|}{6} = \tfrac{3}{6} = \tfrac{1}{2}.
-
Its complement, two ways. An odd face is
E^{c} = \{1, 3, 5\}, so directly
\mathbb{P}(E^{c}) = \tfrac{3}{6} = \tfrac{1}{2}. The complement
rule must agree:
\mathbb{P}(E^{c}) = 1 - \mathbb{P}(E) = 1 - \tfrac{1}{2} = \tfrac{1}{2}. ✓
-
Inclusion–exclusion with overlap. Let
A = \{2, 4, 6\} (even) and
B = \{4, 5, 6\} (at least 4). They
share A \cap B = \{4, 6\}, so
\mathbb{P}(A) = \tfrac{3}{6},
\mathbb{P}(B) = \tfrac{3}{6},
\mathbb{P}(A \cap B) = \tfrac{2}{6}. Inclusion–exclusion
predicts
\mathbb{P}(A \cup B) = \tfrac{3}{6} + \tfrac{3}{6} - \tfrac{2}{6} = \tfrac{4}{6} = \tfrac{2}{3},
and counting directly confirms it:
A \cup B = \{2, 4, 5, 6\} has four outcomes, so
\mathbb{P}(A \cup B) = \tfrac{4}{6} = \tfrac{2}{3}. The
-\,\tfrac{2}{6} removed exactly the double-counted overlap
\{4, 6\}.
As you switch events in the figure, watch how the machinery never changes — the six worlds
and the sixty-four questions are fixed once and for all; only which question you are asking
moves. That is the probability-space discipline in miniature.