Standard Error
The sampling distribution of the mean has a spread, and that spread has a name: the
standard error. It is just the standard deviation of the sample mean
\bar{x} — a measure of how much \bar{x}
wobbles from sample to sample:
\operatorname{SE} = \frac{\sigma}{\sqrt{n}}.
Don't confuse it with \sigma. The population standard deviation
\sigma measures how spread out the individual values are;
the standard error measures how spread out the averages are. Averaging always tames the
wobble, so \operatorname{SE} < \sigma whenever
n > 1.
Precision grows with √n, not n
The \sqrt{n} in the denominator carries a sober warning about
diminishing returns. Because it is the square root — not
n itself — that drives the shrinkage, precision improves slowly:
\operatorname{SE} = \frac{\sigma}{\sqrt{n}} \;\Longrightarrow\; \text{to halve the SE you must } \textbf{quadruple } n.
Going from n=100 to n=400 — four times the
data, four times the cost — only cuts the standard error in half. Each extra digit of precision
gets dramatically more expensive.
Feel the diminishing returns
The faint curve is the population, with spread \sigma. The bold curve
is the distribution of \bar{x}, with spread
\operatorname{SE} = \sigma/\sqrt{n}. Raise
n and the bold curve narrows — but notice you have to go from
n=4 to n=16, a fourfold jump, just to
halve its width.
- The standard error \operatorname{SE} = \sigma/\sqrt{n} is the standard deviation of \bar{x}.
- It measures how much the sample mean wobbles — smaller SE means a more precise estimate of \mu.
- It is not \sigma: \sigma is the spread of single values, SE the spread of averages.
- Precision improves with \sqrt{n}: to halve the SE you must quadruple n.