Independence
Ask someone what "independent" means and you'll get something like "the two things have
nothing to do with each other." That intuition is a decent compass — and a terrible
definition. In probability, independence is not a vibe, a story, or a claim about causation.
It is one precise equation, and either it holds or it doesn't:
\mathbb{P}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).
Getting this wrong is not an academic slip. Nearly every large financial blow-up has, near its
core, an independence assumption that quietly failed: portfolios priced as "a thousand separate
coin flips" turned out to be one coin flipped a thousand times. This page pins down what the
equation says, what it buys you when it holds (products of expectations, additive variances,
diversification), and the classic ways people misread it.
The definition — and why it's a definition
Two events are independent when knowing that one happened tells you nothing
about the other. The clean way to say this is multiplicative — the chance of both is the
product of the chances:
\mathbb{P}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).
This is not a theorem; it is the definition. Whenever the joint probability
factorises like this, the two events carry no information about one another. Notice what the
definition does not say: nothing about the events being physically unrelated, nothing
about different experiments, nothing about cause and effect. Two events on the same
die roll can be independent (we'll build such a pair below). The only test is arithmetic:
compute all three probabilities and check whether the product works out. That makes
independence wonderfully checkable — and dangerously easy to assume without checking.
A picture: the unit square
Let the whole sample space \Omega be the unit square, so area
is probability — throw a dart uniformly at the square and the chance of landing in a
region is exactly its area. Make A a vertical strip of width
\mathbb{P}(A) and B a horizontal strip of
height \mathbb{P}(B). Their overlap A \cap B
is a rectangle, and its area is the product of the side lengths:
\text{area}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).
Drag the sliders: however wide or tall you make the strips, the overlap's area is always the
product — that is independence drawn in one rectangle. The geometry also shows why:
learning "the dart landed in B" squashes your world down to the
horizontal strip, but within that strip the fraction occupied by
A is still exactly \mathbb{P}(A).
Dependence would look like a diagonal or L-shaped region — one whose horizontal extent changes
as you move up the square, so knowing y reshapes your bet about
x.
Worked example: check it on a die
Independence is a computation, so let's actually compute one — on a single roll of a
fair die, where intuition says everything should be tangled together. Take
- A = "the roll is even" = \{2, 4, 6\},
- B = "the roll is at most 4" = \{1, 2, 3, 4\}.
Step 1 — the marginals.
\mathbb{P}(A) = \tfrac36 = \tfrac12 and
\mathbb{P}(B) = \tfrac46 = \tfrac23.
Step 2 — the joint. A \cap B = \{2, 4\}, so
\mathbb{P}(A \cap B) = \tfrac26 = \tfrac13.
Step 3 — the product test.
\mathbb{P}(A)\,\mathbb{P}(B) \;=\; \tfrac12 \cdot \tfrac23 \;=\; \tfrac13 \;=\; \mathbb{P}(A \cap B). \;\checkmark
Independent — on the same die roll. Knowing the roll is at most 4 leaves the evens
exactly half of what's left (\{2,4\} out of
\{1,2,3,4\}), so the information is worthless for betting on
evenness. "Same experiment" does not mean "dependent".
Now break it. Keep A = even but swap in
C = "the roll is at least 4" = \{4, 5, 6\},
so \mathbb{P}(C) = \tfrac12. The joint is
A \cap C = \{4, 6\}, giving
\mathbb{P}(A \cap C) = \tfrac13. But the product is
\mathbb{P}(A)\,\mathbb{P}(C) \;=\; \tfrac12 \cdot \tfrac12 \;=\; \tfrac14 \;\ne\; \tfrac13.
The product test fails, so A and C are
dependent: big rolls are disproportionately even
(\{4,5,6\} has two evens out of three). Two pairs of events, one
die, opposite verdicts — and only the arithmetic could tell you which was which.
The conditional view: news that changes nothing
There is a second, equivalent way to say the same thing. Recall that conditioning on
B means renormalising the world to the cases where
B happened:
\mathbb{P}(A \mid B) = \mathbb{P}(A \cap B) / \mathbb{P}(B). Divide
both sides of the independence equation by \mathbb{P}(B) (assuming
\mathbb{P}(B) > 0) and you get
\mathbb{P}(A \mid B) \;=\; \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)} \;=\; \frac{\mathbb{P}(A)\,\mathbb{P}(B)}{\mathbb{P}(B)} \;=\; \mathbb{P}(A).
Independence means the news B doesn't move your
probability of A. On the die: with
B = "at most 4",
\mathbb{P}(A \mid B) \;=\; \frac{1/3}{2/3} \;=\; \frac12 \;=\; \mathbb{P}(A),
the update is a no-op — exactly the independence we verified. With
C = "at least 4",
\mathbb{P}(A \mid C) \;=\; \frac{1/3}{1/2} \;=\; \frac23 \;\ne\; \frac12,
the news lifts the chance of an even roll from \tfrac12 to
\tfrac23 — dependence, in the language a trader actually uses:
does this information change my price? The two forms are the same statement; the
product form is symmetric in A, B and needs no
\mathbb{P}(B) > 0, which is why it is taken as the definition.
These three confusions are so common they deserve their own warning label:
-
Disjoint is the opposite of independent. If
A and B are mutually exclusive
(A \cap B = \varnothing) and both have positive probability, then
\mathbb{P}(A \cap B) = 0 while
\mathbb{P}(A)\,\mathbb{P}(B) > 0 — the product test fails as badly
as it possibly can. Disjoint events are maximally dependent: if one occurs, the
other is certainly ruled out — the most informative news there is. "They can't happen
together" and "they have nothing to do with each other" are near-opposite statements.
-
Pairwise independence does not imply mutual independence. Three events can
pass the product test two at a time and still be rigidly linked all together — the XOR
construction below is the classic example. Checking pairs is not checking the collection.
-
Zero correlation is strictly weaker than independence. Independence kills
every functional relationship; zero covariance only kills the linear one.
Y = X^2 can be uncorrelated with X
while being a deterministic function of it (worked out below). A risk model that says
"correlation zero, therefore independent, therefore variances add and tails multiply" has
smuggled in an enormous unearned assumption.
Independent random variables
Random variables X, Y are independent when their joint
distribution factorises into the marginals — for every x, y,
\mathbb{P}(X \le x,\, Y \le y) \;=\; F_X(x)\, F_Y(y), \qquad f_{X,Y}(x,y) = f_X(x)\, f_Y(y).
In other words, every event you can build from X is
independent of every event you can build from Y — event
independence, upgraded to hold across two whole families of events at once. The headline
consequence — the one that powers risk and portfolio calculations — is that independence lets
expectations and variances split:
\mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y], \qquad \operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y).
See it on the smallest possible example: two coin payoffs. Flip two fair,
independent coins. Let X pay 1 if the
first coin lands heads (else 0), and Y
pay 1 if the second does. Then
\mathbb{E}[X] = \mathbb{E}[Y] = \tfrac12. The product
XY pays 1 only when both coins
land heads — probability \tfrac14 — so
\mathbb{E}[XY] \;=\; 1 \cdot \tfrac14 \;=\; \tfrac14 \;=\; \tfrac12 \cdot \tfrac12 \;=\; \mathbb{E}[X]\,\mathbb{E}[Y]. \;\checkmark
Now break the independence by wiring the payoffs to the same coin: set
Y = X. The marginals are unchanged
(\mathbb{E}[Y] = \tfrac12), but
XY = X^2 = X, so
\mathbb{E}[XY] = \tfrac12 \ne \tfrac14. Same marginal
distributions, completely different joint behaviour — the marginals alone never determine the
joint. Neither headline identity is obvious, and both are worth deriving: the product rule
first, then the variance of a sum.
The product rule: 𝔼[XY] = 𝔼[X] 𝔼[Y]
Do the cleanest case — discrete (simple) X, Y — where every step is
visible. Say X takes values x_i and
Y takes values y_j. Independence of the
random variables means every pair of value-events factorises:
\mathbb{P}(X = x_i,\, Y = y_j) \;=\; \mathbb{P}(X = x_i)\,\mathbb{P}(Y = y_j).
Write the product XY through the discrete-expectation formula, summing
the value x_i y_j over the joint probability of landing there:
\mathbb{E}[XY] \;=\; \sum_{i}\sum_{j} x_i\, y_j\, \mathbb{P}(X = x_i,\, Y = y_j).
Substitute the factorised joint probability:
=\; \sum_{i}\sum_{j} x_i\, y_j\, \mathbb{P}(X = x_i)\,\mathbb{P}(Y = y_j).
Each term is a product of an "i piece" and a
"j piece", so the double sum splits into a product of two single
sums (distributivity, regrouping the finite sum):
=\; \Big(\sum_{i} x_i\, \mathbb{P}(X = x_i)\Big)\Big(\sum_{j} y_j\, \mathbb{P}(Y = y_j)\Big).
But each bracket is just a marginal expectation, so
=\; \mathbb{E}[X]\,\mathbb{E}[Y].
The general (continuous) case is identical in spirit: replace the factorised PMF by the
factorised density f_{X,Y}(x,y) = f_X(x)\,f_Y(y) and the double sum
by a double integral, and Fubini's theorem splits
\iint x\,y\,f_X(x)f_Y(y)\,dx\,dy into the product of the two
one-dimensional integrals \mathbb{E}[X]\,\mathbb{E}[Y].
Variance of a sum, line by line
Now the additive-variance rule. Set S = X + Y and start from the
computational form \operatorname{Var}(S) = \mathbb{E}[S^2] - (\mathbb{E}[S])^2.
Expand the second moment first, multiplying out (X+Y)^2 and using
linearity:
\mathbb{E}[(X + Y)^{2}] \;=\; \mathbb{E}[X^{2} + 2XY + Y^{2}] \;=\; \mathbb{E}[X^{2}] + 2\,\mathbb{E}[XY] + \mathbb{E}[Y^{2}].
Expand the squared mean by linearity of \mathbb{E}[S] = \mathbb{E}[X] + \mathbb{E}[Y]:
(\mathbb{E}[X + Y])^{2} \;=\; (\mathbb{E}[X] + \mathbb{E}[Y])^{2} \;=\; (\mathbb{E}[X])^{2} + 2\,\mathbb{E}[X]\,\mathbb{E}[Y] + (\mathbb{E}[Y])^{2}.
Subtract the second line from the first, lining the terms up:
\operatorname{Var}(X + Y) \;=\; \big(\mathbb{E}[X^{2}] - (\mathbb{E}[X])^{2}\big) + \big(\mathbb{E}[Y^{2}] - (\mathbb{E}[Y])^{2}\big) + 2\big(\mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y]\big).
The first two brackets are \operatorname{Var}(X) and
\operatorname{Var}(Y); the third is twice the
covariance:
=\; \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y), \qquad \operatorname{Cov}(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y].
This identity holds for any X, Y. Now invoke independence:
by the product rule just proved, \mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y],
so the covariance — and with it the entire cross term — vanishes:
\operatorname{Cov}(X, Y) \;=\; \mathbb{E}[X]\,\mathbb{E}[Y] - \mathbb{E}[X]\,\mathbb{E}[Y] \;=\; 0,
leaving the clean additive law:
\operatorname{Var}(X + Y) \;=\; \operatorname{Var}(X) + \operatorname{Var}(Y).
So variances add precisely when the cross term dies — which independence guarantees, but which
in fact only needs the weaker condition \operatorname{Cov}(X, Y) = 0
(being uncorrelated).
If X and Y are independent (and
integrable), then
\mathbb{E}[XY] \;=\; \mathbb{E}[X]\,\mathbb{E}[Y], \qquad \operatorname{Var}(X + Y) \;=\; \operatorname{Var}(X) + \operatorname{Var}(Y).
The general identity
\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y)
holds for any X, Y; independence forces the covariance to
0, collapsing it to the additive form.
Why quants care: diversification is the variance rule
The additive-variance law is the mathematical engine of diversification.
Suppose you split capital equally across n positions with returns
X_1, \dots, X_n, each with the same variance
\sigma^2. If the positions are genuinely independent, variances add,
and the portfolio return \bar{X} = \tfrac1n \sum X_i has
\operatorname{Var}(\bar{X}) \;=\; \frac{1}{n^{2}} \sum_{i=1}^{n} \operatorname{Var}(X_i) \;=\; \frac{\sigma^{2}}{n},
so the volatility falls like \sigma / \sqrt{n} — a hundred
independent bets are ten times calmer than one. This \sqrt{n} law
is why insurers can promise anything at all, and why a market-making desk can quote thousands
of small trades it would never take singly.
Now watch what a little dependence does. Give every pair of positions a common
correlation \rho > 0 (covariance
\rho\sigma^2). The general variance-of-a-sum identity, with all
\binom{n}{2} cross terms alive, gives
\operatorname{Var}(\bar{X}) \;=\; \frac{\sigma^{2}}{n} + \Big(1 - \frac{1}{n}\Big)\rho\,\sigma^{2} \;\xrightarrow[n \to \infty]{}\; \rho\,\sigma^{2}.
The independent part still melts away, but the correlated part hits a floor of
\rho\sigma^2 that no amount of diversification can remove.
With \rho = 0.1, infinitely many positions are still about a third
as volatile as one (\sqrt{0.1} \approx 0.32). So the entire benefit
of "spreading your bets" rests on the independence assumption — and the cost of assuming
\rho = 0 when the truth is \rho > 0 is a
risk floor you have priced at zero.
Pairwise is not mutual
For three or more events, checking them two at a time is not enough.
Pairwise independence (every pair factorises) is strictly weaker than
mutual independence, which demands that every sub-collection
factorises — including all three together:
\mathbb{P}(A \cap B \cap C) \;=\; \mathbb{P}(A)\,\mathbb{P}(B)\,\mathbb{P}(C).
It is entirely possible for each pair to be independent while the triple is not, so always
say which you mean.
Pairwise ≠ mutual (the XOR example). Flip two fair, independent coins and
let
- A = "the first coin is heads",
- B = "the second coin is heads",
- C = "the two coins differ" (exactly one head) — an XOR.
Each event has probability \tfrac12. Every pair
factorises: for instance A \cap C is "first heads and they differ"
= "heads then tails", probability \tfrac14 = \tfrac12\cdot\tfrac12 = \mathbb{P}(A)\mathbb{P}(C),
and likewise for A, B and for B, C. So
the three are pairwise independent. Yet they are not mutually independent:
knowing A and B fixes
C completely (two heads ⇒ they do not differ), so
\mathbb{P}(A \cap B \cap C) = 0 \;\ne\; \tfrac18 = \mathbb{P}(A)\,\mathbb{P}(B)\,\mathbb{P}(C).
The pairs reveal nothing, but all three together are rigidly linked.
Uncorrelated ≠ independent. The product rule above shows independent
variables are uncorrelated (\operatorname{Cov} = 0), but the
converse fails — covariance only sees linear association. Let
X be uniform on \{-1, 0, 1\} and set
Y = X^{2}. By symmetry \mathbb{E}[X] = 0
and \mathbb{E}[XY] = \mathbb{E}[X^{3}] = 0, so
\operatorname{Cov}(X, Y) \;=\; \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y] \;=\; 0 - 0 \;=\; 0,
they are uncorrelated. But they are about as dependent as can be —
Y is a deterministic function of X.
(Concretely, \mathbb{P}(X = 0,\, Y = 0) = \tfrac13 while
\mathbb{P}(X = 0)\,\mathbb{P}(Y = 0) = \tfrac13\cdot\tfrac13 = \tfrac19,
so the joint does not factorise.) Zero correlation rules out a straight-line
relationship, nothing more.
Largely, yes. The instruments at the centre of the 2008 crisis — CDOs, collateralised debt
obligations — were bundles of thousands of mortgages, sliced into layers. The top "senior"
slice only loses money if a huge fraction of the mortgages default together. Price
that slice under (near-)independence and the arithmetic is intoxicating: if each mortgage
defaults with probability 5\% independently, the chance
that any two default together is 0.05^2 = 0.25\%, three together
0.0125\%, and mass simultaneous default is astronomically
unlikely — the product of many small numbers. Rating agencies duly stamped the senior slices
AAA, the same grade as the safest governments.
The industry-standard tool was the Gaussian copula, popularised by David
X. Li's 2000 paper — a formula summarising the joint behaviour of thousands of mortgages in
a single correlation number, calibrated to placid pre-2007 data. Wired later ran the story as
"Recipe for Disaster: The Formula That Killed Wall Street." The formula wasn't wrong
as mathematics; the near-independence calibration fed into it was wrong as a description of
the world.
Because mortgage defaults share common causes — house prices, unemployment, credit
availability — they are only conditionally rare-together. When US house prices fell
nationally for the first time in the data everyone had calibrated to, the hidden common
factor switched on and defaults arrived in a wave:
\mathbb{P}(A \cap B) turned out to be close to
\mathbb{P}(A), not
\mathbb{P}(A)\,\mathbb{P}(B) — for a
5\% default, a factor-of-20 error, compounding across every pair
in the pool. The "impossible" senior losses happened, and the diversification floor
\rho\sigma^2 from the card above arrived all at once. Traders
have a grim proverb for it: in a crisis, all correlations go to one. The lesson is
not "never assume independence" — it is that
\mathbb{P}(A \cap B) = \mathbb{P}(A)\mathbb{P}(B) is a
quantitative claim about the world, to be tested where it matters most: in the tails.