Overfitting and Underfitting

Every model lives on a dial of complexity, and both ends are bad.

Too simple, too wild, just right

The dots follow a gentle curve with a little noise. Step through three models: a straight line (underfits — misses the curve), a sensible curve (fits the trend), and a wild high-degree polynomial (overfits — threads every point but lurches between them). Watch the test error tell the truth.

The dial you must tune

Model complexity is the master dial of machine learning, and the goal is always the same: turn it to the sweet spot where test error is lowest. Turn too far either way and the model fails — just for opposite reasons. The precise way to think about this balance is the bias–variance tradeoff.