Regression predicts a number. Classification predicts a category: spam or not spam, cat or dog, benign or malignant. The model still learns from labelled examples, but the label is now a class rather than a quantity.

With two features we can picture it: each example is a point, coloured by its class, and the model's job is to draw a boundary that separates the classes. New points are then classified by which side of the boundary they fall on.

Draw the dividing line

Two classes are scattered below. Rotate and shift the boundary to separate them as cleanly as possible; the readout counts how many points it gets right. A line that puts every blue on one side and every orange on the other is a perfect classifier for this data.

The questions ahead

A hard line gives a yes/no answer, but often we want a probability — "85% likely spam." That calls for the sigmoid and logistic regression. And not every boundary is a straight line — but the straight-line case is where all the core ideas live, so we start there.