A classifier rarely says a flat "spam" or "not spam." It outputs a score — a number
between 0 and 1 — and you choose the threshold above which a score counts as
positive. In
The ROC curve answers it by sweeping the threshold across its whole range and plotting what happens, and the AUC — the area under that curve — boils the model's quality down to a single, threshold-free number.
At any threshold, every prediction is a true/false positive/negative, so we can read off two rates from the confusion matrix:
As the threshold drops from 1 toward 0, the model flags more and more examples positive, so both rates climb from 0 up to 1. The ROC curve ("receiver operating characteristic," a name inherited from WWII radar operators) plots TPR on the vertical axis against FPR on the horizontal axis as the threshold slides. Drag the slider and watch the operating point travel up the curve:
A perfect classifier shoots straight up the left edge to (0, 1) — all positives caught, no false alarms — and its AUC is 1. A useless classifier that scores at random lands on the dashed diagonal, where TPR = FPR, with AUC 0.5. Every real model lives in between, and the higher its curve bows toward the top-left corner, the better it is.
The area under the ROC curve ranges from 0.5 (worthless) to 1.0 (perfect). It has a beautifully concrete meaning that has nothing to do with any threshold:
AUC is the probability that the model gives a randomly chosen positive example a
higher score than a randomly chosen negative one:
So an AUC of 0.85 means: pick any real positive and any real negative at random, and 85% of the time the model ranks the positive above the negative. That is why AUC measures how well a model ranks cases, independent of where you eventually put the cut. It is the summary you reach for when the right threshold isn't decided yet, or when the classes are imbalanced and raw accuracy would lie to you.
ROC/AUC has a famous blind spot. The false positive rate divides by the total number of negatives, which is huge when negatives dominate — so a flood of false positives barely nudges the FPR, and the ROC curve can still look excellent even when almost every positive prediction is wrong. In a fraud problem with 1 in 10,000 transactions fraudulent, a model can post an AUC of 0.99 and still bury analysts in false alarms. The fix is the precision–recall curve, which plots precision against recall instead. Because precision divides by the number of predicted positives, it feels every false alarm. Rule of thumb: use ROC/AUC when the classes are roughly balanced or you care about ranking; switch to the precision–recall curve (and its area, "average precision") when positives are rare.
AUC evaluates the model across all thresholds at once, which is exactly why it cannot tell you which threshold to deploy. A model with AUC 0.92 is a good ranker, but you still have to choose an operating point on its curve — high-recall for cancer screening, high-precision for a spam filter — based on the real cost of each mistake. Reporting AUC and then quietly shipping a default 0.5 cutoff is a classic mistake. And beware comparing a raw AUC to a business goal: "0.92" is not "92% accurate," and it is not a probability that any individual prediction is right. It is only the ranking probability above.
A loose field guide, though the "good" bar depends entirely on the problem:
| AUC | Interpretation |
|---|---|
| 0.5 | No better than a coin flip |
| 0.6–0.7 | Weak, but real, signal |
| 0.7–0.8 | Acceptable for many tasks |
| 0.8–0.9 | Strong |
| > 0.9 | Excellent — or a sign of data leakage; check! |
That last row is a genuine warning: an AUC that looks too good to be true usually is, and the culprit
is often information about the answer leaking into the features. We meet that failure mode head-on in
Kaggle Learn's Intermediate
Machine Learning course has you score models and compare them on held-out data in Python, where
ROC-AUC is a one-line call to roc_auc_score. Practise plotting and reading these curves
there on a real competition dataset.