Imagine you're trying to guess the sale price of a house before it's even listed. You'd probably walk around it, jot down a few numbers — how big is it, how many bedrooms, how old is it — and use those numbers to make a guess. A machine learning model does exactly the same thing, just with maths instead of a hunch.
Those measurements you jot down — floor area, bedroom count, age — are the model's features: the inputs, the things you're allowed to look at. The number you're actually trying to guess — the sale price — is the label: the answer. Every supervised learning problem, no matter how fancy it sounds, boils down to this one sentence: given these features, predict this label.
Spam filters guess "spam" or "not spam" from features like word counts and sender info. Weather models guess tomorrow's rainfall from features like today's pressure, humidity and wind. Doctors' diagnostic tools guess a risk score from features like age, blood pressure and test results. Same sentence, every time — only the features and the label change.
Real datasets usually arrive as a table — one row per example, one column per measurement. Here's a tiny slice of a house-price table:
| Floor area (m²) | Bedrooms | Age (years) | Sale price (£) |
|---|---|---|---|
| 72 | 2 | 15 | 210,000 |
| 110 | 3 | 4 | 340,000 |
| 58 | 1 | 40 | 150,000 |
Three of the columns — floor area, bedrooms, age — are features: things you can measure about a house before it sells. The last column, sale price, is the label: the thing nobody knows until the sale actually happens, and the very thing we're building the model to predict. If you were handed this table with the price column blanked out, the features are all you'd have to work with — exactly the position a trained model is in every time it makes a real prediction.
Features and labels don't have to be measurements of physical things, and the label doesn't have to be a price. Take a spam filter. A typical row might look like this:
| Count of "free" | Count of "!" | Sender known? | Label |
|---|---|---|---|
| 3 | 5 | no | spam |
| 0 | 0 | yes | not spam |
| 1 | 2 | no | spam |
Here the features are word counts and a yes/no flag, and the label is one of two categories rather than a number on a sliding scale — this flavour of prediction problem is called classification, and predicting a number like a price is called regression. Either way, the shape of the problem is identical: features go in, a label comes out, and during training the model is shown the correct label so it can learn the pattern that connects the two.
Here's that same idea drawn as a picture instead of a table. Each dot is one example, placed by its two features (feature 1 across, feature 2 up) and coloured by its label. Step through them and the readout names the chosen example's features and its label. Everything a model ever sees about the world arrives in exactly this form — a handful of numbers in, one answer out.
A model is only ever as good as the features it's fed — this is the single most underrated fact in machine learning. Choosing and shaping features, sometimes called feature engineering, is often where the real skill lies: the right feature can make a hard problem trivially easy, while a missing one can make an easy problem impossible.
Go back to the house-price example and ask: would floor area help predict price? Almost certainly — bigger houses reliably cost more, so this feature carries real signal. Now ask: would the colour of the front door help? Almost certainly not — paint colour has essentially nothing to do with market value, so this feature is mostly noise dressed up as data. A skilled feature engineer spends their time hunting for more features like "floor area" and fewer like "front door colour."
Next we'll bundle an example's features into a single tidy object — a
The most dangerous mistake in feature selection isn't picking a useless feature — it's picking one that's too good, because it secretly contains the label. This is called feature leakage, and it silently ruins a model's real-world usefulness even though it can make the model look almost magically accurate while you're building it.
Suppose you're predicting whether a hospital patient will be readmitted within 30 days, and one of your "features" is which discharge-planning code the hospital assigned. Sounds harmless — except that code is only ever assigned after doctors have already decided how risky the patient is. You've accidentally handed the model a feature that encodes the answer. It will score beautifully in testing and then fail completely in the real world, because at the moment a real prediction is needed — when the patient first arrives — that feature simply doesn't exist yet.
The fix is a simple discipline: for every feature, ask "would I actually have this piece of information at the moment I need to make the prediction?" If the honest answer is no, the feature has to go, no matter how well it seems to work.
In the 1990s, researchers built a model to predict which pneumonia patients were at high risk of dying, so hospitals could prioritise the sickest people for admission. The model, trained on real patient records, confidently learned something bizarre: patients with a history of asthma had lower risk than average, and should be sent home.
That's the opposite of medical reality — asthma makes pneumonia more dangerous, not less. What actually happened was feature leakage in disguise: because asthma is known to be dangerous, doctors had always rushed those patients straight to intensive care and treated them aggressively — and that aggressive treatment worked so well that, in the historical data, asthma patients ended up with unusually good outcomes. The model had no idea about "aggressive treatment"; it only saw "asthma" and "survived," and connected the wrong dots. Handing this model real decision-making power would have quietly sent the most vulnerable patients home. It's one of the most cited cautionary tales in all of applied machine learning, precisely because the model's internal accuracy score looked great the whole time.
It's tempting to think "just throw in every feature we can measure, and let the model figure out
what matters." In practice, piling on irrelevant or noisy features often makes a model
worse, not better: with enough useless columns, a model starts finding accidental patterns
in pure coincidence, especially when there isn't a huge amount of data to drown the noise out. As
the number of features climbs, the data needed to tell real patterns from random chance climbs even
faster — a problem with its own name, the
You might assume a recommendation engine like Netflix's runs on an obvious feature list — "what genres you've watched," maybe "your star ratings." In reality, engineers have found that some of the most useful features are far stranger: the exact time of day you press play, whether it's a weekday or weekend, how far into a show you typically watch before quitting, and even how quickly you moved from browsing to pressing play on a particular title. None of these look like "taste" at first glance, yet they turn out to carry real signal about what you'll enjoy next. Good feature hunting often means noticing signals nobody thought to write down on purpose.