Independence

Ask someone what "independent" means and you'll get something like "the two things have nothing to do with each other." That intuition is a decent compass — and a terrible definition. In probability, independence is not a vibe, a story, or a claim about causation. It is one precise equation, and either it holds or it doesn't:

\mathbb{P}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).

Getting this wrong is not an academic slip. Nearly every large financial blow-up has, near its core, an independence assumption that quietly failed: portfolios priced as "a thousand separate coin flips" turned out to be one coin flipped a thousand times. This page pins down what the equation says, what it buys you when it holds (products of expectations, additive variances, diversification), and the classic ways people misread it.

The definition — and why it's a definition

Two events are independent when knowing that one happened tells you nothing about the other. The clean way to say this is multiplicative — the chance of both is the product of the chances:

\mathbb{P}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).

This is not a theorem; it is the definition. Whenever the joint probability factorises like this, the two events carry no information about one another. Notice what the definition does not say: nothing about the events being physically unrelated, nothing about different experiments, nothing about cause and effect. Two events on the same die roll can be independent (we'll build such a pair below). The only test is arithmetic: compute all three probabilities and check whether the product works out. That makes independence wonderfully checkable — and dangerously easy to assume without checking.

A picture: the unit square

Let the whole sample space \Omega be the unit square, so area is probability — throw a dart uniformly at the square and the chance of landing in a region is exactly its area. Make A a vertical strip of width \mathbb{P}(A) and B a horizontal strip of height \mathbb{P}(B). Their overlap A \cap B is a rectangle, and its area is the product of the side lengths:

\text{area}(A \cap B) \;=\; \mathbb{P}(A)\,\mathbb{P}(B).

Drag the sliders: however wide or tall you make the strips, the overlap's area is always the product — that is independence drawn in one rectangle. The geometry also shows why: learning "the dart landed in B" squashes your world down to the horizontal strip, but within that strip the fraction occupied by A is still exactly \mathbb{P}(A). Dependence would look like a diagonal or L-shaped region — one whose horizontal extent changes as you move up the square, so knowing y reshapes your bet about x.

Worked example: check it on a die

Independence is a computation, so let's actually compute one — on a single roll of a fair die, where intuition says everything should be tangled together. Take

Step 1 — the marginals. \mathbb{P}(A) = \tfrac36 = \tfrac12 and \mathbb{P}(B) = \tfrac46 = \tfrac23.

Step 2 — the joint. A \cap B = \{2, 4\}, so \mathbb{P}(A \cap B) = \tfrac26 = \tfrac13.

Step 3 — the product test.

\mathbb{P}(A)\,\mathbb{P}(B) \;=\; \tfrac12 \cdot \tfrac23 \;=\; \tfrac13 \;=\; \mathbb{P}(A \cap B). \;\checkmark

Independent — on the same die roll. Knowing the roll is at most 4 leaves the evens exactly half of what's left (\{2,4\} out of \{1,2,3,4\}), so the information is worthless for betting on evenness. "Same experiment" does not mean "dependent".

Now break it. Keep A = even but swap in C = "the roll is at least 4" = \{4, 5, 6\}, so \mathbb{P}(C) = \tfrac12. The joint is A \cap C = \{4, 6\}, giving \mathbb{P}(A \cap C) = \tfrac13. But the product is

\mathbb{P}(A)\,\mathbb{P}(C) \;=\; \tfrac12 \cdot \tfrac12 \;=\; \tfrac14 \;\ne\; \tfrac13.

The product test fails, so A and C are dependent: big rolls are disproportionately even (\{4,5,6\} has two evens out of three). Two pairs of events, one die, opposite verdicts — and only the arithmetic could tell you which was which.

The conditional view: news that changes nothing

There is a second, equivalent way to say the same thing. Recall that conditioning on B means renormalising the world to the cases where B happened: \mathbb{P}(A \mid B) = \mathbb{P}(A \cap B) / \mathbb{P}(B). Divide both sides of the independence equation by \mathbb{P}(B) (assuming \mathbb{P}(B) > 0) and you get

\mathbb{P}(A \mid B) \;=\; \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)} \;=\; \frac{\mathbb{P}(A)\,\mathbb{P}(B)}{\mathbb{P}(B)} \;=\; \mathbb{P}(A).

Independence means the news B doesn't move your probability of A. On the die: with B = "at most 4",

\mathbb{P}(A \mid B) \;=\; \frac{1/3}{2/3} \;=\; \frac12 \;=\; \mathbb{P}(A),

the update is a no-op — exactly the independence we verified. With C = "at least 4",

\mathbb{P}(A \mid C) \;=\; \frac{1/3}{1/2} \;=\; \frac23 \;\ne\; \frac12,

the news lifts the chance of an even roll from \tfrac12 to \tfrac23 — dependence, in the language a trader actually uses: does this information change my price? The two forms are the same statement; the product form is symmetric in A, B and needs no \mathbb{P}(B) > 0, which is why it is taken as the definition.

These three confusions are so common they deserve their own warning label:

Independent random variables

Random variables X, Y are independent when their joint distribution factorises into the marginals — for every x, y,

\mathbb{P}(X \le x,\, Y \le y) \;=\; F_X(x)\, F_Y(y), \qquad f_{X,Y}(x,y) = f_X(x)\, f_Y(y).

In other words, every event you can build from X is independent of every event you can build from Y — event independence, upgraded to hold across two whole families of events at once. The headline consequence — the one that powers risk and portfolio calculations — is that independence lets expectations and variances split:

\mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y], \qquad \operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y).

See it on the smallest possible example: two coin payoffs. Flip two fair, independent coins. Let X pay 1 if the first coin lands heads (else 0), and Y pay 1 if the second does. Then \mathbb{E}[X] = \mathbb{E}[Y] = \tfrac12. The product XY pays 1 only when both coins land heads — probability \tfrac14 — so

\mathbb{E}[XY] \;=\; 1 \cdot \tfrac14 \;=\; \tfrac14 \;=\; \tfrac12 \cdot \tfrac12 \;=\; \mathbb{E}[X]\,\mathbb{E}[Y]. \;\checkmark

Now break the independence by wiring the payoffs to the same coin: set Y = X. The marginals are unchanged (\mathbb{E}[Y] = \tfrac12), but XY = X^2 = X, so \mathbb{E}[XY] = \tfrac12 \ne \tfrac14. Same marginal distributions, completely different joint behaviour — the marginals alone never determine the joint. Neither headline identity is obvious, and both are worth deriving: the product rule first, then the variance of a sum.

The product rule: 𝔼[XY] = 𝔼[X] 𝔼[Y]

Do the cleanest case — discrete (simple) X, Y — where every step is visible. Say X takes values x_i and Y takes values y_j. Independence of the random variables means every pair of value-events factorises:

\mathbb{P}(X = x_i,\, Y = y_j) \;=\; \mathbb{P}(X = x_i)\,\mathbb{P}(Y = y_j).

Write the product XY through the discrete-expectation formula, summing the value x_i y_j over the joint probability of landing there:

\mathbb{E}[XY] \;=\; \sum_{i}\sum_{j} x_i\, y_j\, \mathbb{P}(X = x_i,\, Y = y_j).

Substitute the factorised joint probability:

=\; \sum_{i}\sum_{j} x_i\, y_j\, \mathbb{P}(X = x_i)\,\mathbb{P}(Y = y_j).

Each term is a product of an "i piece" and a "j piece", so the double sum splits into a product of two single sums (distributivity, regrouping the finite sum):

=\; \Big(\sum_{i} x_i\, \mathbb{P}(X = x_i)\Big)\Big(\sum_{j} y_j\, \mathbb{P}(Y = y_j)\Big).

But each bracket is just a marginal expectation, so

=\; \mathbb{E}[X]\,\mathbb{E}[Y].

The general (continuous) case is identical in spirit: replace the factorised PMF by the factorised density f_{X,Y}(x,y) = f_X(x)\,f_Y(y) and the double sum by a double integral, and Fubini's theorem splits \iint x\,y\,f_X(x)f_Y(y)\,dx\,dy into the product of the two one-dimensional integrals \mathbb{E}[X]\,\mathbb{E}[Y].

Variance of a sum, line by line

Now the additive-variance rule. Set S = X + Y and start from the computational form \operatorname{Var}(S) = \mathbb{E}[S^2] - (\mathbb{E}[S])^2. Expand the second moment first, multiplying out (X+Y)^2 and using linearity:

\mathbb{E}[(X + Y)^{2}] \;=\; \mathbb{E}[X^{2} + 2XY + Y^{2}] \;=\; \mathbb{E}[X^{2}] + 2\,\mathbb{E}[XY] + \mathbb{E}[Y^{2}].

Expand the squared mean by linearity of \mathbb{E}[S] = \mathbb{E}[X] + \mathbb{E}[Y]:

(\mathbb{E}[X + Y])^{2} \;=\; (\mathbb{E}[X] + \mathbb{E}[Y])^{2} \;=\; (\mathbb{E}[X])^{2} + 2\,\mathbb{E}[X]\,\mathbb{E}[Y] + (\mathbb{E}[Y])^{2}.

Subtract the second line from the first, lining the terms up:

\operatorname{Var}(X + Y) \;=\; \big(\mathbb{E}[X^{2}] - (\mathbb{E}[X])^{2}\big) + \big(\mathbb{E}[Y^{2}] - (\mathbb{E}[Y])^{2}\big) + 2\big(\mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y]\big).

The first two brackets are \operatorname{Var}(X) and \operatorname{Var}(Y); the third is twice the covariance:

=\; \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y), \qquad \operatorname{Cov}(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y].

This identity holds for any X, Y. Now invoke independence: by the product rule just proved, \mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y], so the covariance — and with it the entire cross term — vanishes:

\operatorname{Cov}(X, Y) \;=\; \mathbb{E}[X]\,\mathbb{E}[Y] - \mathbb{E}[X]\,\mathbb{E}[Y] \;=\; 0,

leaving the clean additive law:

\operatorname{Var}(X + Y) \;=\; \operatorname{Var}(X) + \operatorname{Var}(Y).

So variances add precisely when the cross term dies — which independence guarantees, but which in fact only needs the weaker condition \operatorname{Cov}(X, Y) = 0 (being uncorrelated).

If X and Y are independent (and integrable), then \mathbb{E}[XY] \;=\; \mathbb{E}[X]\,\mathbb{E}[Y], \qquad \operatorname{Var}(X + Y) \;=\; \operatorname{Var}(X) + \operatorname{Var}(Y). The general identity \operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y) holds for any X, Y; independence forces the covariance to 0, collapsing it to the additive form.

Why quants care: diversification is the variance rule

The additive-variance law is the mathematical engine of diversification. Suppose you split capital equally across n positions with returns X_1, \dots, X_n, each with the same variance \sigma^2. If the positions are genuinely independent, variances add, and the portfolio return \bar{X} = \tfrac1n \sum X_i has

\operatorname{Var}(\bar{X}) \;=\; \frac{1}{n^{2}} \sum_{i=1}^{n} \operatorname{Var}(X_i) \;=\; \frac{\sigma^{2}}{n},

so the volatility falls like \sigma / \sqrt{n} — a hundred independent bets are ten times calmer than one. This \sqrt{n} law is why insurers can promise anything at all, and why a market-making desk can quote thousands of small trades it would never take singly.

Now watch what a little dependence does. Give every pair of positions a common correlation \rho > 0 (covariance \rho\sigma^2). The general variance-of-a-sum identity, with all \binom{n}{2} cross terms alive, gives

\operatorname{Var}(\bar{X}) \;=\; \frac{\sigma^{2}}{n} + \Big(1 - \frac{1}{n}\Big)\rho\,\sigma^{2} \;\xrightarrow[n \to \infty]{}\; \rho\,\sigma^{2}.

The independent part still melts away, but the correlated part hits a floor of \rho\sigma^2 that no amount of diversification can remove. With \rho = 0.1, infinitely many positions are still about a third as volatile as one (\sqrt{0.1} \approx 0.32). So the entire benefit of "spreading your bets" rests on the independence assumption — and the cost of assuming \rho = 0 when the truth is \rho > 0 is a risk floor you have priced at zero.

Pairwise is not mutual

For three or more events, checking them two at a time is not enough. Pairwise independence (every pair factorises) is strictly weaker than mutual independence, which demands that every sub-collection factorises — including all three together:

\mathbb{P}(A \cap B \cap C) \;=\; \mathbb{P}(A)\,\mathbb{P}(B)\,\mathbb{P}(C).

It is entirely possible for each pair to be independent while the triple is not, so always say which you mean.

Pairwise ≠ mutual (the XOR example). Flip two fair, independent coins and let

Each event has probability \tfrac12. Every pair factorises: for instance A \cap C is "first heads and they differ" = "heads then tails", probability \tfrac14 = \tfrac12\cdot\tfrac12 = \mathbb{P}(A)\mathbb{P}(C), and likewise for A, B and for B, C. So the three are pairwise independent. Yet they are not mutually independent: knowing A and B fixes C completely (two heads ⇒ they do not differ), so

\mathbb{P}(A \cap B \cap C) = 0 \;\ne\; \tfrac18 = \mathbb{P}(A)\,\mathbb{P}(B)\,\mathbb{P}(C).

The pairs reveal nothing, but all three together are rigidly linked.

Uncorrelated ≠ independent. The product rule above shows independent variables are uncorrelated (\operatorname{Cov} = 0), but the converse fails — covariance only sees linear association. Let X be uniform on \{-1, 0, 1\} and set Y = X^{2}. By symmetry \mathbb{E}[X] = 0 and \mathbb{E}[XY] = \mathbb{E}[X^{3}] = 0, so

\operatorname{Cov}(X, Y) \;=\; \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y] \;=\; 0 - 0 \;=\; 0,

they are uncorrelated. But they are about as dependent as can be — Y is a deterministic function of X. (Concretely, \mathbb{P}(X = 0,\, Y = 0) = \tfrac13 while \mathbb{P}(X = 0)\,\mathbb{P}(Y = 0) = \tfrac13\cdot\tfrac13 = \tfrac19, so the joint does not factorise.) Zero correlation rules out a straight-line relationship, nothing more.

Largely, yes. The instruments at the centre of the 2008 crisis — CDOs, collateralised debt obligations — were bundles of thousands of mortgages, sliced into layers. The top "senior" slice only loses money if a huge fraction of the mortgages default together. Price that slice under (near-)independence and the arithmetic is intoxicating: if each mortgage defaults with probability 5\% independently, the chance that any two default together is 0.05^2 = 0.25\%, three together 0.0125\%, and mass simultaneous default is astronomically unlikely — the product of many small numbers. Rating agencies duly stamped the senior slices AAA, the same grade as the safest governments.

The industry-standard tool was the Gaussian copula, popularised by David X. Li's 2000 paper — a formula summarising the joint behaviour of thousands of mortgages in a single correlation number, calibrated to placid pre-2007 data. Wired later ran the story as "Recipe for Disaster: The Formula That Killed Wall Street." The formula wasn't wrong as mathematics; the near-independence calibration fed into it was wrong as a description of the world.

Because mortgage defaults share common causes — house prices, unemployment, credit availability — they are only conditionally rare-together. When US house prices fell nationally for the first time in the data everyone had calibrated to, the hidden common factor switched on and defaults arrived in a wave: \mathbb{P}(A \cap B) turned out to be close to \mathbb{P}(A), not \mathbb{P}(A)\,\mathbb{P}(B) — for a 5\% default, a factor-of-20 error, compounding across every pair in the pool. The "impossible" senior losses happened, and the diversification floor \rho\sigma^2 from the card above arrived all at once. Traders have a grim proverb for it: in a crisis, all correlations go to one. The lesson is not "never assume independence" — it is that \mathbb{P}(A \cap B) = \mathbb{P}(A)\mathbb{P}(B) is a quantitative claim about the world, to be tested where it matters most: in the tails.