The Cost Function

Show three people the same scatter of dots and ask them to draw the "best fit" line through it, and you'll get three slightly different lines. One tilts a bit steeper, one sits a bit lower — and every one of them will insist theirs is best. "Best" is a feeling until we pin it down with a number. Training a hypothesis line needs exactly that: one precise, computable number that says how wrong a candidate line is, so two lines can be compared honestly instead of by eye.

That number is the cost function. It looks at every single training example at once, boils the whole dataset's worth of mistakes down into a single score, and the entire job of training is nothing more mysterious than making that one number as small as possible.

Think of a weather app trying to predict tomorrow's temperature from today's, or a shop trying to predict how many ice creams it will sell from the day's forecast high. Neither can ever be perfect — the real world is noisy — but some prediction lines are clearly more useful than others. The cost function is what turns "clearly more useful" from a vibe into arithmetic: it's the same yardstick applied to every candidate line, so whichever line scores lowest wins, no arguing required.

Building the formula, one step at a time

Don't try to swallow the formula whole — build it from three small, sensible steps.

  1. For each example, find the error. Take what the hypothesis predicts, h(x_i), and subtract the real answer, y_i. If the line guesses too high, the error is positive; too low, and it's negative.
  2. Square that error. Squaring does two jobs at once: it makes every error positive (so a guess that's too high and one that's too low both count as "bad", instead of cancelling out), and it punishes big misses much harder than small ones — an error of 4 becomes 16, while an error of 1 stays 1.
  3. Average over every example. Add up all those squared errors and divide by n, the number of training examples, so the score doesn't just grow because the dataset is bigger.

Put the three steps together and you get the mean squared error:

J(w, b) = \frac{1}{n}\sum_{i=1}^{n}\big(h(x_i) - y_i\big)^2.

This J is the cost (sometimes called the loss). Low cost means a good fit; the goal of training is to search for the parameters w and b that make J as small as it can possibly be. The same "punish big misses harder" philosophy behind the square shows up again, in a different costume, when we get to classification's cross-entropy loss — squaring isn't the only way to build a cost function, but the instinct to penalise confident wrongness more than a near-miss runs right through the whole subject.

Worked example: scoring a line by hand

Four students report how many hours they studied for a quiz (x) and the score they got out of 10 (y):

(x, y) = (1, 3),\ (2, 4),\ (3, 7),\ (4, 8).

Try the hypothesis h(x) = 2x (so w = 2, b = 0) and score it example by example:

Add the squared errors — 1+0+1+0=2 — and average over the n=4 examples:

J(2, 0) = \frac{2}{4} = 0.5.

A cost of 0.5 is our scorecard for this exact line. On its own that number means little — it only becomes useful once we compare it with another line's score.

Same data, a worse line

Now try h(x) = 2x + 2 — same slope, but shifted up by 2 — on the exact same four students:

J(2, 2) = \frac{1+4+1+4}{4} = \frac{10}{4} = 2.5.

2.5 is a much higher cost than the first line's 0.5 — and that's exactly the point. The cost function is a genuine scorecard: whichever candidate line scores lower fits the data better, full stop, no eyeballing required. Training is just an efficient search through every possible (w, b) for the pair with the lowest score.

Notice, too, what the two lines' error patterns reveal. The first line's mistakes were small and went both ways — a touch too low here, dead-on there. The second line's mistakes were all in the same direction — every prediction ran high — and squaring made that systematic drift cost far more than the first line's scattered near-misses. A cost function doesn't just count mistakes; it notices patterns in them.

Errors are literally squares

The diagram below draws that same idea geometrically. Each shaded square has a side equal to one example's error, so its area is that error squared — exactly the quantity the formula sums up. The cost is the total shaded area (averaged). Tilt and shift the line with the sliders: as it fits better the squares visibly shrink, and the cost — the area — drops toward its minimum.

Why square it, and not just take the size of the error?

Squaring isn't the only way to make every error positive — plain absolute value |h(x_i) - y_i| would do that job too, and it wouldn't punish big misses as savagely. So why did squaring become the traditional default?

The honest answer is mathematical convenience with a real payoff: the square is a smooth curve everywhere, with no sharp corner at zero, so it can be differentiated cleanly at every point. That matters enormously once we start hunting for the lowest point of the cost — the technique called gradient descent needs to ask "which way is downhill from here?" at every single spot on the cost surface, and a smooth bowl-shaped surface (which squaring guarantees, as the next page shows) answers that question everywhere. Absolute error creates a sharp crease exactly at zero error, which trips up that same downhill search.

A note on units: why the cost looks "too big"

Look back at the worked examples: a line that's typically off by only about 1 point out of 10 on the quiz scored a cost of 0.5, not 1. That's because squaring changes the units — if y is measured in test points, then every squared error, and so the cost itself, is measured in squared test points. A cost of 0.5 "points²" doesn't translate directly back into "half a point of error"; it's a different, harsher scale that leans into big misses on purpose. When people want a number back in the original, friendlier units, they take the square root of the cost — that single tweak is common enough to have earned its own name elsewhere in statistics, but the cost function itself is almost always left un-rooted, because the un-rooted version is the one that behaves nicely under gradient descent.

"Least squares" — minimising the sum of squared errors — feels like a very modern, computer-age idea, but it's over two hundred years old. In 1801 the astronomer Giuseppe Piazzi spotted a new object, the dwarf planet Ceres, then lost it again behind the glare of the Sun after only a few weeks of observations. Where would it reappear?

The young Carl Friedrich Gauss used a method of fitting a curve to Ceres's scattered, noisy sightings by minimising the sum of squared errors — and predicted almost exactly where the asteroid would show up again. It did, right where he said. Adrien-Marie Legendre published the same method first, in 1805, leading to one of mathematics' great priority disputes — but both had landed on the very same formula you just used to score a line of test scores.

Nobody in 1801 had a computer, a calculator, or even a modern theory of probability to justify why squaring the errors was the "right" thing to do — Gauss and Legendre reached it through intuition and by how well it worked in practice, decades before anyone could prove it was the statistically best choice under reasonable assumptions about measurement noise. The formula you now use to score a line of quiz results is, quite literally, the same one that once found a lost world.

Two traps catch almost every beginner the first time they meet the cost function:

Here's a way of looking at machine learning that surprises a lot of people: pick the cost function and you've pretty much picked the task. Regression uses squared error because it cares about "how far off, on average, in real units." Classification tasks reach for cross-entropy loss instead, because they care about "how confidently and correctly did we assign a category." Ranking search results, translating a sentence, or steering a self-driving car all boil down to choosing — and then relentlessly minimising — the cost function that matches what you actually want the model to be good at. Learn to read a cost function, and you're reading the model's entire purpose.

A score you can optimize

Turning "how good is this line?" into a single differentiable number is the move that makes learning possible. Because J(w,b) is a smooth function of the parameters, we can ask which way to nudge w and b to make it smaller — and just keep walking downhill. First, let's see what that cost surface actually looks like.