The name is a trap. Logistic regression sounds like it should predict a number — but it doesn't. It's a classifier: given some features, it answers a yes/no question ("will this email be spam?", "will this loan default?", "will this tumour turn out malignant?") by outputting a probability between 0 and 1. It's the simplest, most interpretable classifier in the toolbox, and usually the first thing a working data scientist reaches for before trying anything fancier.

The trick is to steal the idea straight from linear regression's hypothesis function — a weighted sum of the features — and then bend the result through an S-shaped curve so it can never leave the range [0, 1]. Take the familiar linear score

then pass it through the sigmoid function \sigma to squash it into a probability:

That's the whole model — linear regression's engine, with a sigmoid bolted on the end. You decide the actual class by a threshold — usually 0.5: probability above it, predict class 1; below it, predict class 0.

Why not just use a straight line?

It's tempting to skip the sigmoid entirely: fit an ordinary straight-line regression to points labelled 0 and 1, and just threshold the line at 0.5. For very tidy data it can even look like it works. But a raw line has no floor and no ceiling — a student who studies for a hundred hours would get a "probability" of passing far above 1, which is meaningless, and a single extreme outlier point can drag the whole line (and its threshold crossing) badly off course, flipping predictions for points nowhere near the outlier.

The sigmoid fixes both problems at once. It squeezes every possible score into the honest range (0, 1), and it does its squeezing gently in the middle but flattens out hard at the extremes — so a handful of far-flung points barely move the boundary, while the points actually near the decision are the ones that shape it. That single design choice is why logistic regression behaves so much better than "linear regression plus a threshold" ever could.

A probability curve through the data

Here's the shape of the idea. The points sit at height 0 (class 0, on the left) or 1 (class 1, on the right). The S-curve is the model's predicted probability h(x) as the single feature x varies. Slide its steepness and position so it rises through the gap between the classes — where it crosses 0.5 is the decision threshold, the point of maximum uncertainty between the two classes.

Notice what the sigmoid buys you that a raw linear score never could: no matter how far left or right x goes, h(x) can never sneak below 0 or above 1. A plain straight line has no such promise — it happily predicts a "probability" of 4.7 or -2, which makes no sense for a chance of something happening.

A few reference points are worth memorising, because they show up constantly when reading a trained model: \sigma(-2) \approx 0.12, \sigma(-1) \approx 0.27, \sigma(0) = 0.5 exactly, \sigma(1) \approx 0.73, and \sigma(2) \approx 0.88. Notice the symmetry around the midpoint: moving one unit above zero and one unit below zero land you the same distance from 0.5 in opposite directions — a reflection of the sigmoid's own symmetry, \sigma(-z) = 1 - \sigma(z).

Worked example: will they pass the exam?

Suppose a (already-trained) logistic model predicts whether a student passes an exam from a single feature, hours studied x, with learned weight w = 0.9 and bias b = -4.5:

A student who studies x = 6 hours:

A 71\% chance of passing — comfortably above the 0.5 threshold, so the model predicts pass. Now a student who only studies x = 2 hours:

Only a 6\% chance — the model predicts fail. Somewhere in between, at exactly x = 5 hours, z = 0 and p = 0.5 — the model is a coin flip. That single crossing point is the model's decision boundary for this feature.

Reading the weights: predicting customer churn

Businesses use exactly this model to predict whether a customer is about to churn (cancel a subscription). Say a trained model uses three behaviour features — months since the customer's last purchase (x_1), number of support complaints filed (x_2), and years as a loyal customer (x_3) — with learned weights:

Read the signs before you even plug in numbers. The weight on complaints, +0.7, is positive — every extra complaint pushes z up, and so pushes the predicted probability of churning higher. The weight on loyalty years, -0.5, is negative — every extra year of loyalty pushes z down, pulling the predicted probability of churning lower. This is the single most useful habit in reading a trained logistic model: the sign of a weight tells you the direction a feature pushes the prediction, and its size tells you how hard.

Now plug in a real customer: 3 months since their last purchase, 2 complaints, and 1 year of loyalty:

A 52.5\% churn probability — just barely over the threshold. The model flags this customer as a likely churn risk, but only just; a support team reading this would treat it as "worth a phone call," not "certain to leave." A customer sitting right on the boundary like this is exactly the case a good business process should double-check rather than trust blindly.

How it learns

Logistic regression is trained exactly like its linear cousin: define a cost, then run gradient descent. The right cost here isn't squared error, though — it's cross-entropy, which rewards confident correct answers and savagely punishes confident wrong ones. With it, logistic regression is the reliable baseline classifier of all of machine learning: fast to train, cheap to run, and honest about what it does and doesn't know.

One more practical wrinkle: with many features (imagine a churn model with fifty behaviour signals instead of three), it's easy for the weights to overfit — chasing quirks of the training data instead of the real pattern. The usual fix is regularization, which gently discourages any single weight from growing too large, keeping the model's interpretation honest even as the feature list grows.