Bayes' Theorem

Your doctor calls: the screening test for a rare disease has come back positive. The test is a good one — 99% accurate. Sitting in the waiting room, what would you guess is the chance you actually have the disease? Most people — and, alarmingly, many doctors — answer "about 99%." The real answer is closer to 50%, and sometimes far lower.

That gap between the gut answer and the true answer is not a trick. It is the single most useful warning in all of probability, and Bayes' theorem is the exact rule that gets it right. It tells you how to update a belief when new evidence arrives — and again and again, it overturns intuition.

The rule: flip the conditional

Bayes' theorem rewrites a conditional probability by swapping what is given for what is unknown. You often know P(D \mid H) — how likely the evidence is if the hypothesis holds — but what you actually want is P(H \mid D), the chance the hypothesis holds given the evidence. The theorem flips one into the other:

P(H \mid D) = \frac{P(D \mid H)\,P(H)}{P(D)}.

Read it as a flow of belief about a hypothesis H after seeing data D:

In words: posterior ∝ likelihood × prior. This one proportionality is the backbone of all the Bayesian reasoning ahead — and the reason the likelihood deserves a page of its own.

Worked example: the disease that isn't there

Let's do the waiting-room problem for real. A disease affects 1 in 100 people (prevalence 1%). The test is 99% accurate: if you have the disease it flags positive 99% of the time, and if you are healthy it correctly says negative 99% of the time (so it false-alarms just 1%). You test positive. What is P(\text{disease} \mid +)?

The cleanest way to see it is natural frequencies — forget the percentages and imagine a crowd of 10,000 people walking through the clinic:

Out of 10,000 people… Test says + Test says −
100 actually have it 99 (true positives) 1
9,900 are healthy 99 (false alarms) 9,801

Look at the "Test says +" column. It holds 99 true positives and 99 false positives — an even split. So of everyone who tests positive, only

P(\text{disease}\mid +) = \frac{99}{99 + 99} = \frac{99}{198} = \tfrac{1}{2} = 50\%.

The 1% false-alarm rate sounds tiny, but it acts on the 9,900 healthy people — a huge crowd — and that manufactures just as many false positives as there are true ones. The very same arithmetic comes out of the formula, expanding the evidence with the law of total probability:

P(H \mid D) = \frac{P(D\mid H)\,P(H)}{P(D\mid H)\,P(H) + P(D\mid \lnot H)\,P(\lnot H)} = \frac{0.99\cdot 0.01}{0.99\cdot 0.01 + 0.01\cdot 0.99} = \tfrac12.

Worked example: which bag did the ball come from?

Bayes isn't only for medicine — it updates any belief. Two identical bags sit on a table. Bag A holds 2 red and 8 blue marbles; Bag B holds 7 red and 3 blue. A friend picks a bag at random (so the prior is 50/50) and draws one marble — it's red. Which bag is more likely?

The likelihoods are P(\text{red}\mid A) = 0.2 and P(\text{red}\mid B) = 0.7. With equal priors, Bayes gives

P(B \mid \text{red}) = \frac{0.7 \cdot 0.5}{0.7\cdot 0.5 + 0.2\cdot 0.5} = \frac{0.35}{0.45} = \frac{7}{9} \approx 78\%.

One red marble shoves belief from a 50/50 shrug to nearly 4-to-1 in favour of Bag B. Draw another red, and you can feed this posterior back in as the new prior and update again — belief accumulates, one piece of evidence at a time. That is exactly how a spam filter grows more certain with every suspicious word it reads.

How the prior steers the posterior

The curve is the posterior P(H\mid D) as a function of the prior P(H), for a test with sensitivity P(D\mid H) and false-positive rate P(D\mid\lnot H). The faint diagonal is "no update". Set the test's accuracy and read off the surprise: when the prior is small (left edge), even a strong positive leaves the posterior low. Drag the prior up and the posterior climbs — the very same test becomes trustworthy once the condition is common.

The classic mistake is to ignore the prior — the base rate, how common the thing is to begin with — and read the test's accuracy as if it were the answer. It isn't. Two traps to pin down:

The fix is never to forget the base rate. This is why doctors re-test, and why a positive on a rare-disease screen means "look closer", not "you have it".

Bayes' theorem was published after its author was already dead. The Reverend Thomas Bayes (1701–1761), a Presbyterian minister and amateur mathematician, never published it himself — the result sat unnoticed in his papers until his friend Richard Price found it, tidied it up, and read it to the Royal Society in 1763, two years after Bayes had died.

Then it sat in relative obscurity for the better part of 200 years. But that quiet rule for updating belief turned out to be the engine of the modern world: it powers spam filters, medical diagnosis, and machine learning; and during the Second World War, Alan Turing and the codebreakers at Bletchley Park used Bayesian reasoning to help crack the German Enigma cipher. An 18th-century clergyman's afterthought now quietly runs much of the code around you.

See it explained