A hospital tries a new drug on a hundred patients. Sixty of them get better. Wonderful — but wait. On this illness, roughly half of all patients recover on their own anyway, drug or no drug. So the real question is not "did patients improve?" It is: did the drug actually do anything, or did this batch of patients just get lucky?

That single question — real effect, or random luck? — is asked millions of times a year, about medicines, teaching methods, web-page tweaks, crop yields, and coin tosses. Hypothesis testing is the formal machinery statistics built to answer it. It is, quite deliberately, a courtroom: it starts by assuming nothing is going on, and asks whether the evidence is strong enough to overturn that assumption beyond reasonable doubt.

A hypothesis test is a courtroom for a claim. We put a default position on trial and ask whether the data give us enough reason to overturn it. The two competing claims are:

• the null hypothesis H_0 — "no effect", the status quo, the thing we assume true until shown otherwise;
• the alternative hypothesis H_1 — the claim of a real effect, what we suspect instead.

For a population mean, a typical pair is H_0: \mu = \mu_0 \qquad\text{versus}\qquad H_1: \mu \ne \mu_0. Like a defendant, H_0 is presumed innocent. The burden of proof is on the data.

The logic: assume, then be surprised

The move at the heart of every test is this. Assume H_0 is true. Under that assumption, the sampling distribution of the mean is known — it is a bell centred on \mu_0. Now compute a test statistic from the data and ask: how surprising is a value this extreme, if H_0 really holds?

• A value near the centre is unremarkable — exactly what H_0 predicts.
• A value far out in a tail would be a rare fluke under H_0.

If the observed statistic lands far enough into the tail, we judge the data too surprising to square with H_0, and we reject it in favour of H_1. Exactly how surprising is captured by a single number, the p-value, which you will meet next.

Setting up the two hypotheses

Half the skill of testing is just writing down H_0 and H_1 correctly. The rule of thumb never changes: H_0 is always the boring "no difference" claim, and H_1 is the interesting thing you actually hope to show. Work through three:

• Is this coin fair? The "nothing going on" claim is that it lands heads half the time. So H_0: p = 0.5 versus H_1: p \ne 0.5. You flip it 100 times and get 63 heads. Is 63 a surprising haul if the coin really is fair? That is the whole test.
• Does a new teaching method beat the old one? The status quo is that it makes no difference to scores: H_0: \mu_{\text{new}} = \mu_{\text{old}}. The hope is H_1: \mu_{\text{new}} > \mu_{\text{old}}. Notice the direction — here we only care if the new method is better, a one-sided alternative.
• Did the drug work? H_0: the recovery rate is the same as the natural rate, 0.5. H_1: it is higher. Sixty of a hundred recovering is a bit above 0.5 — but is it far enough above to rule out plain luck?

In every case the recipe is identical: assume the boring claim, then measure how startled we should be by the data we actually got.

How surprising is the data?

Below is the null sampling distribution — the bell we would see if H_0 were true, drawn in z-units (standard deviations from \mu_0). Slide the observed statistic and watch where it falls. Out near the centre it is business as usual; pushed far into a tail it becomes a value H_0 rarely produces — the kind of surprise that argues against H_0.

Innocent until proven guilty

The courtroom analogy is not a loose metaphor — it is exact, and worth taking apart carefully:

• H_0 is the presumption of innocence. The court starts by assuming the defendant did nothing wrong, just as the test starts by assuming there is no effect.
• The data are the evidence. A jury does not convict on a hunch; it needs evidence strong enough to be surprising if the defendant were truly innocent.
• "Reject H_0" is a guilty verdict. The evidence was too improbable under innocence, so we overturn the presumption.
• "Fail to reject H_0" is "not guilty" — not "innocent". It says the evidence fell short of the bar, nothing more.

And, just like a real trial, the burden of proof sits entirely on one side. We do not ask the defendant to prove innocence; we ask the prosecution to prove guilt. In a test we never ask the data to prove H_0; we only ask whether they can overturn it.

We never prove H₀

A test has only two verdicts: reject H_0, or fail to reject H_0. The second is not a proof of H_0 — exactly as "not guilty" is not the same as "innocent". It means only that the data were not surprising enough to convict. Absence of evidence against H_0 is not evidence that H_0 is true.

See it explained