The Central Limit Theorem
Imagine you know nothing about a population except that it is a mess. Household incomes: a huge
crowd near the low end and a long thin tail of millionaires. The number of pips on a rolled die:
six flat, equal bars. The weight of pumpkins at a fair: lumpy and bimodal. These shapes have
nothing in common. Yet do one thing to each — draw samples and average them — and every
single one of those histograms of averages collapses into the same shape: the
smooth, symmetric bell of the normal
distribution.
That is the central limit theorem (CLT), arguably the single most astonishing
result in all of statistics. Take a population of any shape — skewed, lumpy, spiky, it
genuinely does not matter. Draw a sample of size n and record its mean
\bar{x}. As n grows, the distribution of
\bar{x} becomes approximately normal — no matter how
un-bell-like the population was. It is why the bell curve shows up everywhere, and why nearly every
statistical method you will ever meet works at all.
What it promises
Combine the central limit theorem with what we already know about
the sampling
distribution of the mean. For large n,
\bar{x} \;\approx\; N\!\left(\mu,\ \frac{\sigma^2}{n}\right):
it is centred at \mu, has
standard error
\sigma/\sqrt{n}, and — the genuinely new part — is approximately
normal in shape. The centre and the spread were already true for any
sample size; the theorem adds the third fact, about the shape, and it is the shape that is
the miracle. And notice: the promise is about the distribution of \bar{x},
not about the individual values, which keep the population's original messy shape forever.
How large is "large"? A common rule of thumb is n \gtrsim 30. The more
skewed or lumpy the population, the larger the n you need before the bell
settles in; for a population already close to normal, even a tiny n will
do. Thirty is a rough guide, not a law — think of it as "usually plenty for the everyday amount of
skew you meet."
Worked example 1 — averaging dice
A single die is the opposite of a bell: roll it many times and you get six equal bars, a perfectly
flat distribution. Its mean is \mu = 3.5 and its
standard deviation is \sigma \approx 1.71.
Now average n = 2 dice. The total ranges from 2 to 12, but there is only
one way to make a total of 2 (a mean of 1) and six ways to make a total of 7 (a mean of 3.5), so the
middle already bulges — the flat bars have become a little triangle. Average
n = 5 dice and the histogram of the mean is already a recognisable bell,
centred at 3.5 with standard error
\sigma/\sqrt{5} \approx 1.71/2.24 \approx 0.76. Average
n = 30 and it is a tight, gorgeous normal curve with standard error
1.71/\sqrt{30} \approx 0.31. Nothing about the die changed — we only
averaged, and the bell appeared.
Worked example 2 — heavily skewed incomes
Household income is famously lopsided: a dense pile of ordinary earners and a long right tail
stretching out to the very rich. Suppose the population has mean
\mu = \$60{,}000 and standard deviation
\sigma = \$40{,}000. A single household tells you almost nothing — it
could be a student on \$8{,}000 or a banker on
\$800{,}000.
Take a random sample of n = 100 households and compute the mean income.
By the CLT, that sample mean behaves like
N\!\left(60{,}000,\ (40{,}000/\sqrt{100})^2\right), i.e. a bell centred
at \$60{,}000 with standard error
40{,}000/\sqrt{100} = \$4{,}000. Because income is very skewed,
n = 100 is doing real work here — at n = 10 the
distribution of the mean would still lean noticeably right. Big skew, bigger
n: the theorem still delivers, it just takes longer to get there.
Worked example 3 — why we get to use z-scores
Here is the payoff that makes the CLT the workhorse of statistics. Because
\bar{x} is approximately normal, we may apply every normal-based tool to
it — even when the raw data is nowhere near normal. Using the income example, what is the chance a
sample of 100 households has a mean above \$68{,}000?
Standardise with the standard error, not the raw \sigma:
z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}} = \frac{68{,}000 - 60{,}000}{4{,}000} = 2.
The empirical
rule tells us about 95% of a normal distribution sits within two standard deviations of
the centre, so only about 2.5% lies above z = 2. There is roughly a
2.5% chance the sample mean exceeds \$68{,}000 — a
genuine, usable probability, computed from skewed income data, entirely on the strength of the CLT.
Watch the bell emerge
The faint curve is a deliberately skewed population — lopsided, with a long
right tail. The bold curve is the distribution of the sample mean
\bar{x}. At n=1 it echoes the skew; raise
n and it pulls into a symmetric, narrow bell centred
at \mu, with width \sigma/\sqrt{n} — exactly
as the theorem promises. Notice two things happening at once: the shape becomes symmetric, and the
bell gets narrower (the 1/\sqrt{n} squeeze). Both are the CLT
at work.
- For large n, the distribution of \bar{x} is approximately normal.
- This holds whatever the population's shape — skewed, bimodal, anything (with finite variance).
- The bell is centred at \mu with standard error \sigma/\sqrt{n}.
- Rule of thumb: n \gtrsim 30 is usually enough; this is why the normal appears everywhere.
This is the single most muddled point about the CLT, so pin it down now. The theorem is about the
distribution of the sample mean \bar{x} — not
about the individual data values. Collecting more data does not make your raw
numbers turn into a bell curve.
Skewed income data stays skewed forever. Survey a million households instead of a hundred and the
histogram of individual incomes is still lopsided with a long rich tail — you have simply
drawn a sharper picture of the same skewed shape. What becomes normal is the distribution of the
average of a sample, imagined over many possible samples. "More data makes it bell-shaped"
is false; "the averages of samples become bell-shaped" is the truth. Confuse the two and every
conclusion downstream goes wrong.
Very nearly. In the 1800s, statisticians kept finding the same bell curve wherever they looked —
the heights of soldiers, the errors astronomers made measuring a star, the sizes of crops. It
seemed almost supernatural that one shape should govern so much of nature, and some
thinkers came close to treating it as a mystical law of the universe.
The CLT is the down-to-earth reason. Anything that is the sum or average of many small,
independent effects ends up bell-shaped. A person's height is thousands of tiny genetic
and nutritional nudges added together; a measurement error is many little independent slips piled
up. Add enough small random things and you cannot help but get a bell. That is the deep, unifying
reason the normal distribution is everywhere — not magic, but arithmetic.
You can even watch it happen physically. A Galton board (or "quincunx") drops
balls through a triangle of pegs; each peg nudges a ball left or right at random, and the pile at
the bottom — the sum of all those tiny independent nudges — always heaps up into a bell. It is the
central limit theorem you can hold in your hands.
See it explained