The Sampling Distribution of the Mean

Here is the conceptual leap that makes all of statistical inference work. You almost never get to measure a whole population — every voter, every light bulb, every tree in the forest. You get one sample, you compute its mean, and somehow you have to say how much that single number can be trusted. That feels impossible: how can one sample tell you how wrong it might be?

The trick is to imagine what you would see if you could repeat the whole experiment over and over. Draw a sample of size n, compute its mean \bar{x}, write it down. Now do it again — a fresh sample, a fresh mean — and again, thousands of times. Collect all those means into their own histogram. That histogram is the sampling distribution of the mean, and its shape is the key that unlocks how far a single sample's mean can stray from the truth.

We already know the sample mean \bar{x} varies — a fresh sample gives a fresh value. So treat \bar{x} as a quantity with a distribution of its own. Imagine repeating the experiment many times: draw a sample of size n, record its mean, and do it again and again. Collecting all those means gives the sampling distribution of the mean.

Centred, but tighter

Two facts shape this distribution. First, it is centred at \mu: averaging doesn't push \bar{x} high or low, so its long-run mean is exactly the population mean,

\mathbb{E}[\bar{x}] = \mu.

Second, it is narrower than the population. A single observation can be extreme, but an average of n of them washes out the extremes — highs and lows cancel. The spread of \bar{x} is

\operatorname{sd}(\bar{x}) = \frac{\sigma}{\sqrt{n}},

which shrinks as n grows. Bigger samples give means that huddle more tightly around \mu. (That shrinking spread has its own name, the standard error, which comes next.)

Three pictures to hold in your head

1. Two heights of people vs two class averages. Pick two random adults and their heights might differ by 30 cm — one is 155 cm, the other 185 cm. But take the average height of two whole classrooms of 30 children each, and those two class averages will barely differ — maybe 1 cm apart. Individuals scatter widely; averages of many individuals scatter narrowly. The sampling distribution is the distribution of those averages.

2. Why bigger samples pin down the mean. Suppose a population has \sigma = 10. With n = 4 the sample means spread out with width 10/\sqrt{4} = 5. Bump the sample to n = 100 and that width becomes 10/\sqrt{100} = 1 — five times tighter. Every extra observation you average in cancels a little more of the noise, so the means crowd closer and closer to \mu. (Exactly how that \sqrt{n} shrinkage plays out, and why the shape becomes a bell no matter what the population looks like, are the jobs of the standard error and the central limit theorem.)

3. Individual spread vs mean spread, side by side. Lay the raw population curve and the curve of sample means over the same centre. The raw data is a broad hill of width \sigma; the means form a sharp spike of width \sigma/\sqrt{n} sitting right on top of it. Same centre, wildly different widths — that gap is the whole point.

Watch it tighten

The wide faint curve is the population. The bold curve is the distribution of the sample mean \bar{x}. Both sit over the same centre \mu, but as you raise n the bold curve pulls in tight — its width is \sigma/\sqrt{n}. The mean of a large sample is a far more reliable pointer to \mu than a single value.

The "sampling distribution of the mean" is a distribution of means, not of individual data values — and mixing these two up is one of the most damaging mistakes in all of statistics.

If you report the population's spread when you meant the spread of the mean, you overstate your uncertainty enormously; do it the other way round and you claim a false precision. A poll of 1,000 people whose ages have \sigma = 20 years does not have a mean uncertain by 20 years — it's uncertain by about 20/\sqrt{1000} \approx 0.6 years. Confusing the two turns a razor-sharp estimate into a useless one, or vice versa.

Here is the beautiful sleight of hand at the heart of statistics. In real life you never actually draw thousands of samples — you have exactly one: one poll, one experiment, one batch of measurements. The sampling distribution describes what would happen if you repeated the study endlessly, an experiment nobody ever performs.

And yet that purely hypothetical distribution is what lets a single telephone poll of just 1,000 people announce a "±3% margin of error" for an entire nation of millions. The theory of the imaginary repeated experiment hands you a number for how much your one real sample can be trusted. It is one of the most counterintuitive and powerful ideas humans have ever had: you quantify the uncertainty of a thing you did once by reasoning about the thing you never did.

See it explained