Type I and Type II Errors
A verdict in a
hypothesis test
can be wrong in two different ways — and they are not the same mistake. Lay the truth against
our decision:
-
Type I error — we reject a true H_0. A
false positive: we cry "effect!" when there is none. Its long-run rate is
exactly the significance level, \alpha.
-
Type II error — we fail to reject a false
H_0. A false negative: a real effect slips past
us. Its rate is written \beta.
The two correct verdicts are rejecting a false H_0 and failing to
reject a true one.
Power: catching a real effect
If \beta is the chance of missing a real effect, then
1 - \beta is the chance of catching it. This is the
power of the test:
\text{power} = 1 - \beta = \mathbb{P}(\text{reject } H_0 \mid H_1 \text{ true}).
A powerful test rarely lets a genuine effect through. So a good test wants both
\alpha small (few false alarms) and power high (few misses).
The two errors, side by side
The left bell is the sampling distribution under H_0;
the right bell is the distribution under H_1 (a real
effect shifts the mean across). The vertical line is the decision threshold: we
reject H_0 for anything to its right.
-
The α region is the slice of the null bell to the right of the
threshold — a true H_0 that we wrongly reject.
-
The β region is the slice of the alternative bell to the left of
the threshold — a false H_0 that we wrongly keep.
The trade-off
The two errors pull against each other. Slide the threshold right to shrink
\alpha (fewer false positives) and the \beta
slice of the alternative bell grows — more misses, less power. Slide it left and
the trade reverses. With fixed sample size you cannot drive both errors to zero at once; choosing
\alpha is choosing where on this see-saw to sit. (Collecting more data
is what separates the bells and eases the squeeze.)
- Type I — reject a true H_0 (false positive), rate \alpha.
- Type II — fail to reject a false H_0 (false negative), rate \beta.
- Power = 1 - \beta — the chance of detecting a real effect.
- Lowering \alpha raises \beta — a trade-off you can only escape with more data.