Take every possible choice of a line's parameters, compute the cost J for each one, and plot the result. Do that for linear regression's mean squared error and something beautiful happens: no matter what dataset you feed it, the resulting cost landscape is always a smooth, single-bottomed bowl. There are no bumps, no false valleys to get trapped in — just one true lowest point.

That single fact is what makes training a line so friendly, and it's the picture that turns the abstract idea of "minimise the cost" into something you can literally see: training a model is just rolling a ball downhill until it settles at the bottom of that bowl.

It's worth pausing on how strange that is. The cost function itself never mentions a bowl, a hill, or a ball — it's just a sum of squared numbers. The bowl shape is a hidden geometric consequence of that formula, one that only reveals itself once you plot cost against parameter value. Spotting that hidden shape is exactly what turns "guess and check" into a real optimisation strategy.

One knob at a time: a parabola

Hold the bias b fixed and plot J against just the weight w. Because the error is squared, the resulting curve is a parabola — the same U-shaped curve you'd get from y = w^2 — with exactly one lowest point, the best weight for that fixed bias.

Let's see that U-shape appear from real numbers, not just imagine it.

Worked example: watching the U-shape form

Recall the four students from the cost function page: hours studied x = 1, 2, 3, 4 against test scores y = 3, 4, 7, 8. To keep the arithmetic simple, force the line through the origin — set the bias to b = 0 — so there's only one knob left to turn, the weight w, and the hypothesis is h(x) = wx.

Compute the cost J(w) by hand at a handful of candidate weights:

• w = 1: predictions 1, 2, 3, 4 → squared errors 4, 4, 16, 16 → J(1) = 40/4 = 10.
• w = 1.5: predictions 1.5, 3, 4.5, 6 → squared errors 2.25, 1, 6.25, 4 → J(1.5) = 13.5/4 = 3.375.
• w = 2: predictions 2, 4, 6, 8 → squared errors 1, 0, 1, 0 → J(2) = 2/4 = 0.5.
• w = 2.5: predictions 2.5, 5, 7.5, 10 → squared errors 0.25, 1, 0.25, 4 → J(2.5) = 5.5/4 = 1.375.
• w = 3: predictions 3, 6, 9, 12 → squared errors 0, 4, 4, 16 → J(3) = 24/4 = 6.

Lay those five numbers out — 10,\ 3.375,\ 0.5,\ 1.375,\ 6 — and the U-shape jumps out even before you plot a single point: the cost falls steeply, bottoms out somewhere near w = 2, then climbs again. That's no coincidence — it's exactly the best-fitting weight found by hand on the very same dataset back on the cost function page. Slide the weight in the diagram below and watch the dot trace that same curve continuously.

Reading the bowl

The bottom of the bowl is the weight that fits the data best; anywhere else, up the sides, the line is worse and the cost is higher. The steepness of the wall right beside the dot is a hint about which way to go to make the cost smaller — walk downhill, in the direction the slope points — which is exactly the idea the next page turns into a precise algorithm, gradient descent.

Notice something else the shape guarantees: the further the dot sits from the bottom, the steeper the wall tends to be, and the closer it gets, the flatter the curve becomes — right at the very bottom the curve is momentarily flat, neither rising nor falling. That flatness at the minimum isn't a coincidence; it's the mathematical signature of "this is as good as it gets", and it's exactly the condition an algorithm can test for to know when to stop searching.

From one knob to two (and to millions)

With both w and b free, the bowl becomes a 3-D valley over the (w, b) plane — same idea, one more dimension. Slice that 3-D bowl with a horizontal plane at some fixed cost value and look straight down from above, and you get a contour map: a set of nested oval rings, each ring a curve of equal cost, exactly like the rings on a topographic map joining points at equal altitude. The very centre of the innermost ring is the minimum.

Real models have millions of parameters, so their "bowl" lives in a million-dimensional space no one can actually picture. But the principle never changes: find the lowest point of the cost surface, however many dimensions it happens to have.

See the bowl in 3-D

Here is that two-parameter cost J(w, b) as a genuine rotatable surface — a bowl sitting over the whole (w, b) plane. Drag it to turn it in space and look into the valley from any angle. The single low point, marked below, is where both knobs are tuned together to give the best-fitting line's (w, b); every other spot on the sheet is a worse line and a higher cost.

Worked example: nudging the second knob

The one-parameter table above fixed b = 0 and only moved w — which finds the best line among lines through the origin, not necessarily the best line overall. Set w = 2 (the winner from before) and try nudging b too, on the same four students:

• h(x) = 2x - 1: squared errors work out to 4, 1, 4, 1 → J(2, -1) = 10/4 = 2.5.
• h(x) = 2x: squared errors 1, 0, 1, 0 → J(2, 0) = 2/4 = 0.5.
• h(x) = 2x + 0.5: squared errors 0.25, 0.25, 0.25, 0.25 → J(2, 0.5) = 1/4 = 0.25.

Sliding b from -1 up to 0.5, with w held at 2 the whole time, the cost fell from 2.5 to 0.5 to a new best of 0.25 — lower than anything the one-parameter search found. That's the two-parameter bowl in action: moving along the b direction found an even better spot than the best point along the w-only slice. The true minimum of the full bowl sits somewhere that tunes both knobs together, not either one alone.

This is exactly why the full two-parameter search matters in practice: fixing one knob and perfecting the other only ever finds the best point on that one slice of the bowl, like exploring a single street instead of the whole city. A search that's free to move diagonally — adjusting w and b together, a little of each at a time — can cut straight toward the true bottom instead of creeping along one axis, then the other, then back again.