Real data can have hundreds or thousands of features — far too many to visualise, slow to compute with, and riddled with redundancy (height in centimetres and in inches carry the same information twice; a shopper's total spend and their number of orders usually move together too). Dimensionality reduction squeezes the data down to a few features while keeping as much of its real structure as possible — the way a good summary compresses a whole book into a tight paragraph without losing the plot.

The key idea is projection: cast the high-dimensional points down onto a lower-dimensional surface — like a 3-D object throwing a 2-D shadow. Choose the surface well and the shadow keeps the data's important shape; choose badly and you flatten away what mattered. A basketball held up to a lamp can throw a neat, round shadow or a long, thin sliver, depending only on the angle you hold it at — the ball itself never changes, only how much of its true shape survives the projection. Dimensionality reduction is about hunting for the angle whose shadow keeps the most of the ball.

Data secretly likes fewer dimensions

Here's the surprising fact that makes dimensionality reduction possible at all: many real high-dimensional datasets secretly live near a much lower-dimensional structure. A photograph made of a million pixel values sounds hopelessly complicated, but neighbouring pixels are heavily correlated — skies are mostly sky-coloured, skin is mostly skin-coloured — so a much smaller set of "meta-features" (rough shapes, colour blocks, gradients) can capture almost everything that matters about the image. The million raw numbers are mostly redundant restatements of a far simpler underlying picture, and dimensionality reduction is the tool that finds that simpler picture.

The same story plays out for hand-written digits. A small greyscale image of a handwritten "7" might be recorded as 784 separate pixel-brightness numbers (a 28-by-28 grid), which sounds like 784 dimensions of information. But every genuine "7" shares the same rough skeleton — a top bar and a diagonal stroke — so the enormous space of "all possible 784-number grids" barely gets used; real handwritten digits cluster onto a much smaller, twisty surface tucked inside that huge space. Dimensionality reduction is the search for that hidden surface and a sensible set of coordinates on it.

Casting a 2-D cloud onto a line

Here a 2-D cloud is projected down to a 1-D line — two features compressed into one. Rotate the line and watch the points' shadows land on it. Some angles keep the points nicely spread out; others squash them on top of each other, destroying the structure. The spread retained is the readout to maximise.

See the pancake: 3-D points hugging a 2-D sheet

High-dimensional data often really lives near a lower-dimensional sheet. Here is a cloud of points in three dimensions — but look closely and they barely leave a tilted, flattened plane embedded in the space (drawn faintly through the cloud). Drag to rotate and you'll catch the angle where the whole cloud collapses to a thin pancake. Because the points hug that sheet, two coordinates along it capture almost everything and the third — the thin direction across it — is nearly redundant. Dimensionality reduction's whole job is to find that sheet and keep it.

A worked example: almost one dimension already

Take five 2-D points that happen to sit almost exactly on the diagonal line y = x: (1, 1.1), (2, 2.2), (3, 2.9), (4, 4.1) and (5, 4.8). Project each point onto the diagonal direction (1,1)/\sqrt2 by adding its two coordinates and dividing by \sqrt2: the five projected positions come out at roughly 1.49, 2.97, 4.17, 5.73, 6.93. Their spread (variance) along that line works out to about 3.73.

Now project the very same points onto the perpendicular direction instead — subtract the coordinates rather than add them. The five values barely differ from each other at all, and their variance collapses to about 0.01. Almost the entire spread of the original data lives along the diagonal; almost none of it lives across it. So replacing each point's two coordinates with a single number — its position along the diagonal — throws away barely any information: two dimensions have become one, for a tiny cost. Try dragging the angle slider above toward the cloud's own natural direction and away from it, and watch the reported variance rise and fall the same way. Finding that "best line" automatically, for data with any number of dimensions, is exactly what principal component analysis does — using the eigenvectors you met in linear algebra.

Why bother? A 50-feature dataset you can actually look at

Suppose a dataset describes each customer with 50 different measurements — age, spend, visit frequency, dozens more. A human being cannot look at a 50-dimensional scatter plot; we can barely manage three axes on a screen, and comfortably read only two. Reduce those 50 features down to just 2 well-chosen new ones, though, and suddenly the whole dataset becomes an ordinary scatter plot a person can scan in seconds, spotting clusters, outliers and trends that were completely invisible buried in 50 columns of numbers. This is one of the most common everyday uses of dimensionality reduction: not to feed a model, but simply to let a human being see the data at all.

It shows up before training a model too. A dataset with 50 raw features but only, say, 2,000 example rows can make many models struggle — there's a lot of room for a model to latch onto coincidental patterns in so many dimensions with comparatively few examples to check them against. Reduce those 50 features down to a well-chosen 8 or 10 first, and the very same model often trains noticeably faster and generalises better to new data it hasn't seen, simply because it has fewer, richer numbers to make sense of instead of fifty thin, partly-redundant ones.

Keep the spread

A good projection keeps the points as spread out as possible — that spread, the variance, is what carries the information. The line that retains the most variance is the best 1-D summary of the data, and finding it for any number of dimensions is exactly what principal component analysis does. Beyond simplicity, throwing away redundant, noisy dimensions this way often gives three very practical wins: models train faster on fewer numbers per example, the data becomes easy to visualise, and — because noisy or repetitive features can actively confuse a model — accuracy sometimes even improves. It's the same lesson as features and labels' warning that more features isn't automatically better: quality and non-redundancy matter more than raw quantity.