Bagging, Boosting and Stacking

One model is a single opinion. An ensemble is a committee — many models whose combined verdict beats any member. You already met one committee in random forests, where hundreds of trees vote. But "combine many models" is a whole design space, and there are three classic ways to run the committee, each fixing a different weakness. Understanding which one to reach for is really a question about the bias–variance character of your base model.

The three families are bagging (train models in parallel and average them), boosting (train models in sequence, each repairing the last), and stacking (train a small extra model to learn how to combine the others). Parallel, sequential, or learned — that is the whole taxonomy.

Parallel vs sequential, at a glance

The deepest split is between bagging, where the base models are grown independently and never see each other, and boosting, where each model is built specifically to fix what the previous ones got wrong. Bagging's models could be trained on a thousand machines at once; boosting's must wait in line.

Stacking sits on top of either: a first layer of diverse base models makes predictions, and a second-layer "meta-model" learns the best way to blend those predictions into a final answer.

Bagging: average away the variance

Bagging — short for bootstrap aggregating — trains many copies of the same kind of model, each on a different random bootstrap sample (drawn with replacement) of the data, then averages their predictions (or takes a majority vote). Because each model sees a slightly different dataset, each overfits to slightly different quirks, and averaging cancels those quirks out.

Through the bias–variance lens, bagging is a variance-reduction machine. It works best on base models that are individually low-bias but high-variance — flexible learners that fit the signal well but wobble wildly from sample to sample. Deep decision trees are the textbook case, which is exactly why the random forest is a bagged ensemble of deep trees. Averaging leaves the (already low) bias untouched while crushing the variance:

\mathrm{Var}\!\left(\tfrac{1}{n}\sum_i M_i\right) \approx \frac{\mathrm{Var}(M)}{n} \quad\text{(if the models were independent).}

Boosting: chip away at the bias

Boosting flips the strategy. It trains models one at a time in sequence, and each new model is told to concentrate on the examples the current committee still gets wrong — by re-weighting those hard cases, or by fitting the leftover errors directly. The models are then added together, so the ensemble slowly bends itself toward the truth.

Boosting's natural base model is the opposite of bagging's: a weak learner that is high-bias but low-variance, such as a shallow "stump" of a tree that is barely more than a coin flip. On its own such a learner is hopeless, but a long sequence of them, each patching the residual mistakes of the last, combines into a powerful predictor. So boosting is a bias-reduction machine — it manufactures a low-bias committee out of high-bias parts. Its most famous incarnation, gradient boosting, is the subject of gradient-boosted trees, and it dominates competitions on tabular data.

Stacking: let a model learn the blend

Bagging and boosting combine many copies of one kind of model with a fixed rule (average, or weighted sum). Stacking is greedier: it takes a handful of deliberately different models — say a random forest, a boosted-tree model, a logistic regression and a nearest-neighbour model — and trains a second-layer meta-model to learn the best way to weigh their predictions against each other.

The intuition is that different model families make different kinds of mistakes: where the forest stumbles, the logistic model may be confident, and vice versa. The meta-model learns to trust each base model exactly where it tends to be right. Because it exploits the diversity of genuinely different learners, a good stack can shave off both bias and variance — at the cost of real complexity and a serious risk of leakage if you build it carelessly (see the warning below).

The three, side by side

BaggingBoostingStacking
How models trainIn parallel, independentlyIn sequence, each fixes the lastBase models in parallel, then a meta-model
Base modelLow-bias, high-variance (deep trees)High-bias, low-variance (weak learners)Several diverse model families
Mainly reducesVarianceBiasBoth (via diversity)
Combined byAveraging / votingWeighted sumA learned meta-model
FlagshipRandom forestGradient-boosted treesCompetition-winning blends

The classic stacking blunder is to train the base models on the training set, ask them to predict that same training set, and feed those predictions to the meta-model. But the base models have already seen those rows, so their in-sample predictions are unrealistically good — the meta-model learns to trust them far more than it should, and the whole stack collapses on real data. The fix is out-of-fold predictions: split the training data into folds, and for each row use only base models that were not trained on that row to generate its meta-features. This is the same leakage discipline as cross-validation, applied one level up. Skip it and your stack will look brilliant in development and fall apart in production.

In 2009 Netflix paid out $1 million to the team that best improved its movie-recommendation engine. The winning entry wasn't a single clever model — it was a gigantic stack, blending hundreds of different predictors into one. It worked, and it made ensembling famous, but there was a punchline: Netflix reportedly never deployed the winning blend, because the engineering cost of running that many stacked models in production outweighed the accuracy gain. It is a perfect cautionary tale — stacking can squeeze out the last drop of accuracy, but that last drop is often too expensive and fragile to ship. Bagging and boosting, being simpler, are far more common in real systems.

Learn this on Kaggle

Kaggle Learn's Intermediate Machine Learning course builds ensembles on real competition data, where boosted trees are the default weapon and blending several models is a standard leaderboard trick. Practise there to feel which ensemble helps which problem.