The Dataset: Train and Test

Before any learning can happen, a model needs examples — cases where somebody already knows the answer. Stack enough of those examples up and you get a dataset: a big table with one row per example and one column per measurement, plus (for supervised learning) one more column holding the label — the answer for that row.

It's tempting to think the clever part of machine learning is the algorithm — the equations, the code, the "AI." In practice, the quality and size of this humble table very often matters more than which fancy algorithm you point at it afterwards. A brilliant algorithm fed a tiny, messy, biased table will lose to a simple algorithm fed a large, clean, representative one, almost every time.

Worked example: a toy dataset

Suppose you want to predict whether a piece of fruit is ripe, from how it looks and feels. Here's a small dataset somebody might collect by picking fruit off a tree and tasting it:

#	Colour score (0–10)	Firmness (0–10)	Days on tree	Ripe?
1	8	3	12	yes
2	2	9	3	no
3	7	4	10	yes
4	3	8	4	no
5	9	2	14	yes
6	4	7	5	no
7	6	5	9	yes

Each numbered line is a row, and each row is one example — one piece of fruit somebody actually inspected. "Colour score," "Firmness" and "Days on tree" are columns, and here they play the role of features: the inputs a model is allowed to look at. "Ripe?" is the final column, the label: the answer that was only knowable by actually tasting the fruit. Seven rows is a real dataset — tiny, but a dataset all the same — and everything a model could ever learn about "ripeness" would have to come from patterns hiding inside exactly these seven rows.

Why more examples usually help

Seven pieces of fruit is barely enough to notice a pattern, let alone trust one. Imagine trying to learn "ripe fruit tends to be colourful and soft" from just two examples — one ripe, one not. You genuinely couldn't tell whether colour mattered, or firmness did, or both, or neither; with so little evidence, almost any rule would "fit" by pure coincidence. Collect seventy examples instead of seven, and the accidental coincidences start to wash out, while the real pattern — riper fruit really is softer and more colourful — stands out clearly above the noise.

This is why serious datasets are often huge: a spam filter might learn from millions of emails, a voice assistant from millions of recorded sentences. More genuine, relevant examples generally make a model's learned pattern more reliable — though, as the vignette below explains, "more" is not the whole story.

Worked example: cleaning a messy dataset

Real-world data is almost never as tidy as the fruit table above. Suppose the dataset actually arrived looking like this:

#	Colour score	Firmness	Days on tree	Ripe?
1	8	3	12	yes
2	2	9	3	no
3	7	—	10	yes
4	3	8	4	no
5	9	2	400	yes
6	4	7	5	no
3	7	—	10	yes

Three problems are hiding here, and spotting them is a routine (and important!) first step before any training begins:

A missing value — row 3's firmness was never recorded. You can't just leave a gap; common fixes are dropping the row, or filling it with a sensible estimate like the average firmness of similar fruit.
An obviously wrong entry — row 5 claims the fruit sat on the tree for 400 days, over a year. That's almost certainly a typo (probably 40, or a slipped decimal point), and left uncorrected it would quietly drag the model's sense of "normal" wildly off course.
A duplicate row — the last row is an exact repeat of row 3, perhaps copied in twice by accident during collection. Left in, it secretly gives that one example double the voting power of every other row.

This unglamorous scrubbing work is often called data cleaning, and experienced practitioners will tell you it regularly eats more hours than choosing or tuning the model itself.

No algorithm, however sophisticated, can rescue a model from a bad dataset. If your fruit examples were all picked from a single tree in one orchard, a model trained on them might completely fail on a different variety of apple grown somewhere else — it never saw anything else, so it never learned anything else. This is called an unrepresentative or biased dataset, and it is one of the most common real-world causes of an ML system quietly failing the people it wasn't trained on.

There's a second, closely related trap: a dataset that's too small for how complicated the problem really is. A handful of examples can always be memorised perfectly, giving a false sense that the model has "learned" something general, when really it has just crammed the exact answers for those exact examples — a failure mode you'll meet properly as overfitting. The short version for now: a dataset earns trust by being both clean and plentiful relative to how hard the pattern is to learn — neither quality nor quantity alone is enough.

For decades, computers were embarrassingly bad at recognising what's in a photo. Then, in the 2000s, researchers built ImageNet: a dataset of over fourteen million photos, each carefully labelled by hand with what it showed — "dog," "kayak," "espresso maker," and thousands of other categories. In 2012, a team trained a new kind of neural network on ImageNet and smashed every previous record for image recognition, almost overnight.

What's easy to miss in that story is that the algorithm wasn't the only star — the dataset was too. Without fourteen million carefully labelled examples to learn from, that same network would have had nothing to learn from at all. ImageNet is now widely credited as one of the sparks that ignited the modern deep-learning boom, and it's a vivid reminder that a great dataset can matter every bit as much as a great algorithm. Self-driving car companies take the same lesson to heart today, deliberately driving millions of real miles specifically to build the training datasets their driving models will need.

Once the dataset is clean: split it before you train

Here's a trap. If you train a model on some data and then test it on the same data, of course it does well — it has effectively seen the answers. That tells you nothing about how it will cope with the future. So, once a dataset is clean and ready, we always split it into two parts:

Training set — the examples the model learns from.
Test set — examples held back, never shown during training, used only to judge the finished model.

The test set is a stand-in for "new data the model has never seen." Performance there is the only honest measure of how good the model really is.

Fit on train, judge on test

The line is fitted using only the training points. Toggle to reveal the test points — held out during training — and see how well the line predicts them. A model that does well on data it never trained on is one you can trust.

The golden rule

Never let the test set leak into training. The moment a model has peeked at the test data — even indirectly, through choices you made — your score is inflated and meaningless. Keeping a clean, untouched test set is the single most important discipline in applied machine learning, and it's what makes the next idea, generalization, measurable at all.