A hospital tries a new blood-pressure drug. Twenty patients take it and drop an average of 11 mmHg; twenty on a placebo drop 8 mmHg. The drug group did better — by 3 mmHg. But so what? Two random groups of people, given nothing but sugar pills, would also differ a little just by luck of the draw. The real question is not "did the groups differ?" (they always do), but "is the new treatment really better, or is the gap just chance?" That is the job of the two-sample test: it weighs the observed gap between two means against the wobble you'd expect from sampling noise alone.

You already know the one-sample t-test: is this mean far from a fixed claim \mu_0? Now there is no printed claim — just two groups, and we ask whether they differ from each other. The shape of the statistic is identical: an observed difference divided by the noise you'd expect in that difference.

The statistic keeps its shape: signal over noise

Call the two group means \bar{x}_1 and \bar{x}_2. The null hypothesis is that the underlying population means are equal, H_0:\ \mu_1 = \mu_2, i.e. their difference is zero. The test statistic is

the observed gap in the numerator over the standard error of that gap below. Read it the same way as before: the top is how far apart the two sample means landed, the bottom is how far apart pure chance would routinely push them. A big |t| means the gap dwarfs the noise.

The one new ingredient is the denominator. Each mean carries its own uncertainty, and when you subtract two independent noisy quantities their variances add (variance of a difference of independent things is the sum of the variances). So

Each term s_i^2/n_i is just the squared standard error of one mean; the difference's noise is the two stacked in quadrature. This "unequal-variance" form is Welch's t-test — the safe default, because it never assumes the two groups are equally spread. Compare the resulting t to a t-distribution (or, for large samples, the standard normal) exactly as in the one-sample case.

A picture: two distributions, and the gap between them

Below are the two groups drawn as bell curves — same shape, but their centres sit a distance apart. Slide the true gap between the population means and the sample size, and watch the sampling distribution of the difference \bar{x}_1 - \bar{x}_2 (the narrow curve, centred on the gap) sharpen. The dashed line marks a difference of zero — the null. When the narrow curve pulls clear of that line, the gap is many standard errors from zero and the test fires; overlap the line and the gap is well within chance.

Notice the lever: raising n doesn't move the two fat curves, but it shrinks \mathrm{SE}_{\text{diff}} = \sqrt{s_1^2/n_1 + s_2^2/n_2}, so the narrow curve tightens and the very same gap becomes easier to call real. Bigger samples turn a fuzzy hint into a firm verdict.

Worked example 1 — drug vs placebo (independent groups)

Two separate sets of patients. The drug group: n_1 = 20, \bar{x}_1 = 11 mmHg, s_1 = 4. The placebo group: n_2 = 20, \bar{x}_2 = 8 mmHg, s_2 = 5. Is the 3 mmHg advantage real? Use Welch.

Step 1 — the standard error of the difference (add the two s^2/n terms):

Step 2 — the test statistic (observed gap over that noise):

Step 3 — decide. With roughly \nu \approx 36 degrees of freedom, a two-sided |t| = 2.10 gives a p-value near 0.043 — just under 0.05. So at \alpha = 0.05 we reject H_0: the drug's edge is a little more than we'd credit to chance. "A little" is the operative word — it's a borderline call, and a confidence interval for the difference makes that honesty explicit (next card).

The confidence interval for the difference

A p-value gives a yes/no verdict; the interval says how big the difference plausibly is. It has the familiar shape — estimate \pm critical value \times standard error — with the estimate now being the gap:

For the drug trial, with t^{\*}\approx 2.03 at 95%: 3 \pm 2.03\times 1.43, i.e. about 3 \pm 2.9, giving [0.1,\ 5.9] mmHg. The interval excludes zero (barely) — the same borderline verdict as the test, since "reject H_0 at 5%" and "the 95% interval misses zero" are two views of one fact. But the interval says more: the true benefit could be as small as a whisker above nothing or as large as 6 mmHg. Significant, yes; overwhelmingly convincing, no.

Paired samples: the same subjects, measured twice

Sometimes the two "groups" aren't separate people at all. Measure each patient's blood pressure before and after treatment; test the same students in September and in June; give each taster both recipes. These are paired (or matched) designs — every observation in group 1 is tied to a specific partner in group 2.

Here the trick is beautiful: collapse each pair into its single difference d_i = \text{after}_i - \text{before}_i, and the two-sample problem evaporates into a one-sample test on those differences. Ask only: is the mean difference \bar{d} far from zero?

where s_d is the standard deviation of the differences and n is the number of pairs. That's just the one-sample t-test you already know, pointed at the column of d_i's.

Worked example 2 — a before/after study (paired)

Eight runners follow a new training plan. For each, we record the drop in their 5 km time (in seconds):

Step 1 — mean of the differences:

Step 2 — spread of the differences: the sample standard deviation works out to s_d \approx 3.34 s, so

Step 3 — the statistic and verdict:

A |t| of 8.5 on 7 degrees of freedom is enormous — a p-value far below 0.001. We emphatically reject H_0: the plan really does make these runners faster. The huge t is no accident of the numbers — it's the reward for pairing, which we unpack next.

Why pairing is more powerful

Runners differ wildly from each other — a club champion and a weekend jogger might be minutes apart. If you ran the study as two independent groups (some runners tested "before-style", different ones "after-style"), that huge person-to-person spread would swell the standard error and drown the training effect. Pairing sidesteps it entirely: each runner is their own control, so the between-person variation cancels out of the difference. What's left in s_d is only the effect of the treatment plus small within-person noise.

The payoff: a much smaller standard error, a larger t, and the power to detect a genuine effect with far fewer subjects. When a paired design is available — same units measured twice, or sensibly matched pairs — use it.

See it explained