The Hypothesis Function
Before a model can be trained, you have to decide its shape. Will
it be a straight line? A curve? Something stranger? That shape — the mathematical form of your
guess — is called the hypothesis function. For simple linear regression with one
feature it is
h(x) = wx + b.
It has two adjustable "knobs", the parameters the model learns:
w, the weight (the slope — how strongly the feature
pushes the prediction), and b, the bias (the intercept —
the prediction when the feature is zero). Training means searching for the values
of w and b that fit the data best — the
hypothesis function itself is just the guess those knobs plug into.
Notice what the hypothesis function does not do: it doesn't look at the data and decide,
on its own, to become a curve instead of a line. A straight line is the shape a human chose for it.
Training only ever turns the two knobs already built into that chosen shape — it never redesigns the
shape itself. Keep that distinction in mind; it's easy to blur and it matters a lot once things
stop fitting well.
The same idea, dressed in different letters
You will meet this line written two different ways, and it helps to recognise both on sight.
Textbooks and everyday language often use h(x) = wx + b, with "weight"
and "bias" as the names. Many machine-learning courses instead write it with the Greek letter
\theta ("theta"):
h_\theta(x) = \theta_0 + \theta_1 x.
It's exactly the same line — \theta_0 plays the role of
b (the intercept), and \theta_1 plays the role
of w (the slope). The subscript just numbers the knobs instead of giving
each one its own letter, which turns out to be far more convenient once a model has hundreds of
knobs instead of two.
There's a neat trick hiding in that \theta_0 notation, too. Some texts
write the hypothesis as h_\theta(x) = \theta_0 x_0 + \theta_1 x_1 with an
invented feature x_0 that is always equal to
1. That folds the bias into the same pattern as every other weight, so
one tidy rule — "multiply each feature by its weight and add them up" — covers the whole line,
intercept included. It looks like unnecessary fuss for one feature, but once a model has dozens of
features,
that uniform pattern is exactly what makes the maths (and the code) scale cleanly.
Worked example: building a prediction table
Suppose training has already settled on w = 3 and
b = 5, so h(x) = 3x + 5. Evaluating it by hand
at a few values of x builds up a small prediction table:
h(0) = 3(0) + 5 = 5,
h(1) = 3(1) + 5 = 8,
h(2) = 3(2) + 5 = 11,
h(3) = 3(3) + 5 = 14,
h(4) = 3(4) + 5 = 17.
Notice the pattern: every step of 1 in x adds
exactly 3 to the prediction — that's the weight doing its job, one unit
of feature at a time. And notice what happens right at x = 0: the weight
term 3(0) vanishes completely, leaving just
h(0) = b = 5. That's precisely why b is
described as "the prediction when the feature is zero" — it's the one row of the table where the
weight has nothing to multiply.
Worked example: how wrong is the guess?
A hypothesis function is only a guess — the real data rarely matches it exactly. Take the
same h(x) = 3x + 5 and compare its predictions against the actual labels
that were recorded for those same inputs:
At x = 1: predicted 8, actual
9, error 1.
At x = 2: predicted 11, actual
10, error -1.
At x = 3: predicted 14, actual
17, error 3.
Those little gaps between prediction and reality are exactly what the
cost
function adds up to score how good w and
b currently are — the hypothesis function makes the guesses, the cost
function grades them. It's worth sitting with the fact that every choice of
w and b produces its own list of errors like
this one. Try w = 4, b = 4 instead and every prediction — and therefore
every error — changes. Training is the search, across all the possible knob settings, for the pair
that makes those errors smallest overall.
Two knobs, every line
Play with the sliders below and watch the two knobs work independently.
b slides the whole line up and down without changing its tilt — set
w = 0 and the line goes perfectly flat at height b.
w tilts the line — make it negative and the line runs downhill instead
of uphill. Between the two of them, they can reach any straight line at all.
Worked example: changing one knob at a time
Start again from h(x) = 3x + 5, and change only
b, from 5 to 8.
Every single prediction shifts by exactly the same amount, +3:
h(0) goes from 5 to
8, and h(4) goes from
17 to 20. The whole line simply slides
upward, keeping its tilt.
Now reset b to 5 and instead change
w, from 3 to 1.
This time h(0) doesn't move at all — it's still
5, since the weight term vanishes at x = 0 —
but h(4) drops all the way from 17 to
9. The line pivots around its intercept, getting shallower. Two knobs,
two completely different jobs.
The form of the hypothesis function — straight line, in this case — is chosen by a
human, before training even starts. The training algorithm never invents that shape; it only ever
searches for the best w and b within
the shape it's been given.
That has a sharp consequence: if the true relationship between x and
y is strongly curved — say it bends like a "U" — then no choice of
w and b will ever make
h(x) = wx + b fit it well. You could train for a thousand years and it
still wouldn't help, because a straight line simply cannot bend into a U shape. Choosing the right
shape of hypothesis function is a decision that happens before any tuning at all — and
getting it wrong dooms the model no matter how good the training algorithm is.
Once you start reading about machine learning, you'll see the letters \theta
and w everywhere, standing for wildly different things from one model to
the next. That's deliberate: they are generic names for
"the numbers we're trying to learn", whatever the model happens to be. In simple
linear regression \theta is just two numbers. In a
neuron
it's a handful. In a large neural network it can be billions.
Get comfortable with this generic naming now, while there are only two knobs to keep track of — it
will reappear, unchanged in spirit, in every model for the rest of this course. If a later page ever
says "the model learns its weights", you'll now know exactly what that means: somewhere inside, a
hypothesis function just like this one is having its knobs turned.
Here's the reassuring secret: every more advanced model you'll meet later —
logistic
regression, a single
neuron,
a whole neural network — is really just a more flexible hypothesis function, with
more knobs, wired together in cleverer patterns. Underneath all of them sits the exact same loop you
just practised by hand: make a guess with the current knobs, measure how wrong it is, adjust the
knobs, repeat.
In fact, choosing the right shape of hypothesis — linear, curved, or something else
entirely — is often more important to a model's success than any fancy trick used to train it. A
perfectly-tuned straight line still loses to a roughly-tuned curve, if a curve is what the data
actually needs.
So when you meet a model with a scary-sounding name later in this course, it can help to ask the
same two questions you're asking here: what shape of hypothesis function is it using underneath, and
what knobs does training get to turn? Strip away the jargon and there's almost always a
"guess-measure-adjust" loop like this one waiting at the centre.
The names matter later
"Weight" and "bias" sound grander than "slope" and "intercept", but it's the same idea — and the
grander names are the ones that scale. A neuron
is exactly h(x) = wx + b with one extra twist, and a network has
millions of weights and biases. Learn the two-knob line cold and you've learned the atom every
bigger model is built from.
One last thing worth banking now: w and b
belong to the model, not to any single example. Once training picks a value for them, that
same w and b get used to predict every input
you ever feed the model — the new student's hours, the shop's next temperature reading, all of it.
Only the input x changes from prediction to prediction; the hypothesis
function itself stays fixed until you decide to retrain it.