The Central Limit Theorem

Imagine you know nothing about a population except that it is a mess. Household incomes: a huge crowd near the low end and a long thin tail of millionaires. The number of pips on a rolled die: six flat, equal bars. The weight of pumpkins at a fair: lumpy and bimodal. These shapes have nothing in common. Yet do one thing to each — draw samples and average them — and every single one of those histograms of averages collapses into the same shape: the smooth, symmetric bell of the normal distribution.

That is the central limit theorem (CLT), arguably the single most astonishing result in all of statistics. Take a population of any shape — skewed, lumpy, spiky, it genuinely does not matter. Draw a sample of size n and record its mean \bar{x}. As n grows, the distribution of \bar{x} becomes approximately normal — no matter how un-bell-like the population was. It is why the bell curve shows up everywhere, and why nearly every statistical method you will ever meet works at all.

What it promises

Combine the central limit theorem with what we already know about the sampling distribution of the mean. For large n,

\bar{x} \;\approx\; N\!\left(\mu,\ \frac{\sigma^2}{n}\right):

it is centred at \mu, has standard error \sigma/\sqrt{n}, and — the genuinely new part — is approximately normal in shape. The centre and the spread were already true for any sample size; the theorem adds the third fact, about the shape, and it is the shape that is the miracle. And notice: the promise is about the distribution of \bar{x}, not about the individual values, which keep the population's original messy shape forever.

How large is "large"? A common rule of thumb is n \gtrsim 30. The more skewed or lumpy the population, the larger the n you need before the bell settles in; for a population already close to normal, even a tiny n will do. Thirty is a rough guide, not a law — think of it as "usually plenty for the everyday amount of skew you meet."

Worked example 1 — averaging dice

A single die is the opposite of a bell: roll it many times and you get six equal bars, a perfectly flat distribution. Its mean is \mu = 3.5 and its standard deviation is \sigma \approx 1.71.

Now average n = 2 dice. The total ranges from 2 to 12, but there is only one way to make a total of 2 (a mean of 1) and six ways to make a total of 7 (a mean of 3.5), so the middle already bulges — the flat bars have become a little triangle. Average n = 5 dice and the histogram of the mean is already a recognisable bell, centred at 3.5 with standard error \sigma/\sqrt{5} \approx 1.71/2.24 \approx 0.76. Average n = 30 and it is a tight, gorgeous normal curve with standard error 1.71/\sqrt{30} \approx 0.31. Nothing about the die changed — we only averaged, and the bell appeared.

Worked example 2 — heavily skewed incomes

Household income is famously lopsided: a dense pile of ordinary earners and a long right tail stretching out to the very rich. Suppose the population has mean \mu = \$60{,}000 and standard deviation \sigma = \$40{,}000. A single household tells you almost nothing — it could be a student on \$8{,}000 or a banker on \$800{,}000.

Take a random sample of n = 100 households and compute the mean income. By the CLT, that sample mean behaves like N\!\left(60{,}000,\ (40{,}000/\sqrt{100})^2\right), i.e. a bell centred at \$60{,}000 with standard error 40{,}000/\sqrt{100} = \$4{,}000. Because income is very skewed, n = 100 is doing real work here — at n = 10 the distribution of the mean would still lean noticeably right. Big skew, bigger n: the theorem still delivers, it just takes longer to get there.

Worked example 3 — why we get to use z-scores

Here is the payoff that makes the CLT the workhorse of statistics. Because \bar{x} is approximately normal, we may apply every normal-based tool to it — even when the raw data is nowhere near normal. Using the income example, what is the chance a sample of 100 households has a mean above \$68{,}000?

Standardise with the standard error, not the raw \sigma:

z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}} = \frac{68{,}000 - 60{,}000}{4{,}000} = 2.

The empirical rule tells us about 95% of a normal distribution sits within two standard deviations of the centre, so only about 2.5% lies above z = 2. There is roughly a 2.5% chance the sample mean exceeds \$68{,}000 — a genuine, usable probability, computed from skewed income data, entirely on the strength of the CLT.

Watch the bell emerge

The faint curve is a deliberately skewed population — lopsided, with a long right tail. The bold curve is the distribution of the sample mean \bar{x}. At n=1 it echoes the skew; raise n and it pulls into a symmetric, narrow bell centred at \mu, with width \sigma/\sqrt{n} — exactly as the theorem promises. Notice two things happening at once: the shape becomes symmetric, and the bell gets narrower (the 1/\sqrt{n} squeeze). Both are the CLT at work.

This is the single most muddled point about the CLT, so pin it down now. The theorem is about the distribution of the sample mean \bar{x}not about the individual data values. Collecting more data does not make your raw numbers turn into a bell curve.

Skewed income data stays skewed forever. Survey a million households instead of a hundred and the histogram of individual incomes is still lopsided with a long rich tail — you have simply drawn a sharper picture of the same skewed shape. What becomes normal is the distribution of the average of a sample, imagined over many possible samples. "More data makes it bell-shaped" is false; "the averages of samples become bell-shaped" is the truth. Confuse the two and every conclusion downstream goes wrong.

Very nearly. In the 1800s, statisticians kept finding the same bell curve wherever they looked — the heights of soldiers, the errors astronomers made measuring a star, the sizes of crops. It seemed almost supernatural that one shape should govern so much of nature, and some thinkers came close to treating it as a mystical law of the universe.

The CLT is the down-to-earth reason. Anything that is the sum or average of many small, independent effects ends up bell-shaped. A person's height is thousands of tiny genetic and nutritional nudges added together; a measurement error is many little independent slips piled up. Add enough small random things and you cannot help but get a bell. That is the deep, unifying reason the normal distribution is everywhere — not magic, but arithmetic.

You can even watch it happen physically. A Galton board (or "quincunx") drops balls through a triangle of pegs; each peg nudges a ball left or right at random, and the pile at the bottom — the sum of all those tiny independent nudges — always heaps up into a bell. It is the central limit theorem you can hold in your hands.

See it explained