Encoding Categorical Variables

Almost every real dataset arrives with columns that are words, not numbers: a colour is "red," a city is "Osaka," a payment method is "credit card." A model built on feature vectors cannot multiply "red" by a weight — arithmetic only speaks numbers. So before any linear model, distance metric, or gradient step can run, every categorical column has to be encoded: turned into numbers in a way that preserves what the category means and adds no meaning that isn't really there.

That last clause is the whole art. The wrong encoding quietly injects a false claim — that "Tuesday" is bigger than "Monday," or that "Tokyo" sits halfway between "Sydney" and "Cairo" — and the model dutifully learns the fiction. This page is a tour of the three encodings you reach for most, and the trap hiding inside each.

One-hot encoding: one column per category

The safest default for an unordered (nominal) category is one-hot encoding: give every distinct value its own column, and mark a 1 in the column that matches, 0 everywhere else. A single "colour" column becomes a little block of 0/1 columns:

colour	colour_red	colour_green	colour_blue
red	1	0	0
green	0	1	0
blue	0	0	1
red	1	0	0

Now each colour is a corner of a coordinate space, all equally far apart — no fake ordering, no fake distances. This is exactly what you want when the categories have no natural rank. The cost is width: k categories become k new columns, and that width bites hard when k is large.

The one-hot columns for a category always sum to 1 across every row (exactly one is hot).
So the last column is a perfect linear combination of the others plus the intercept — the design matrix is collinear, and a linear model's coefficients become unidentifiable.
The fix: drop one level (the "reference" category), leaving k-1 columns that carry the same information.

Tree models and regularised models don't strictly need the drop, but for plain linear/logistic regression it matters: keep all k dummies and an intercept and the maths has no unique solution. Drop one, and each remaining coefficient reads as "this category versus the reference."

Ordinal encoding: numbers for a real order

Some categories do have a rank: small < medium < large, bronze < silver < gold, a survey's strongly-disagree … strongly-agree. For these, ordinal encoding maps them to ordered integers (0, 1, 2, \dots) so the model can use the ordering as signal — one column instead of k.

The danger is using it when there is no order. "Label encoding" — slapping arbitrary integers on unordered categories (red→0, green→1, blue→2) — looks tidy and is a classic beginner mistake: it tells a linear or distance-based model that green is between red and blue, and that blue is "twice" green. Trees can sometimes cope (they only split on thresholds), but for anything geometric this manufactures relationships that don't exist. Ordinal encoding is for ordered categories only.

Target (mean) encoding: powerful, and dangerous

When a category has hundreds or thousands of levels — zip codes, product IDs, user agents — one-hot encoding explodes into a sea of columns and label encoding lies. A sharper tool is target encoding (also called mean encoding): replace each category with the average value of the target for the rows in that category. If 8% of orders from postcode SW1 were fraudulent, encode SW1 as 0.08. One dense, informative column, whatever the cardinality.

Target encoding is built from the target you are trying to predict — so if you compute the per-category means over the whole training set and then let a row use the mean it helped create, you have handed the model a peek at its own label. Validation scores look brilliant and collapse in production. This is textbook data leakage. The disciplined fix: compute the category means only on the out-of-fold data — fit the encoding inside each cross-validation fold, never on the rows it will be applied to — and smooth rare categories toward the global mean so a category seen twice doesn't get a wild, overconfident estimate.

A useful smoothing formula blends the category mean \bar{y}_c (from n_c rows) toward the global mean \bar{y}:

\hat{y}_c = \frac{n_c\,\bar{y}_c + m\,\bar{y}}{n_c + m},

where m is a "prior weight": categories with few rows lean on the global mean, categories with many rows trust their own.

The high-cardinality problem

The three tools line up neatly against how many levels a column has:

Situation	Reach for	Because
Few, unordered levels	One-hot (drop one)	No fake order; width is affordable
Ordered levels	Ordinal	The order is genuine signal
Many levels (high cardinality)	Target / mean encoding	Stays one column; but fit inside CV folds

High cardinality is the recurring villain: a "city" column with 5,000 values becomes 5,000 one-hot columns, most of them almost always zero — a sparse, memory-hungry mess that also invites overfitting because rare cities appear only a handful of times. Target encoding, frequency encoding (replace a category by how often it occurs), or grouping rare levels into an "other" bucket are the usual escapes.

With k mutually exclusive categories, knowing that a row is "not red" and "not green" forces it to be blue — the last dummy is fully determined by the others. That redundancy is precisely the collinearity that breaks a linear solver, and precisely why removing one column costs nothing: the reference category is encoded as "all zeros." The kept coefficients simply become contrasts against that baseline, which is often more interpretable, not less.

Learn this on Kaggle

Kaggle Learn's Feature Engineering course walks through one-hot, ordinal and target encoding on real competition data in Python — including how to build target encoding safely so it doesn't leak. Practise the tradeoffs there on a dataset with genuinely high-cardinality columns.