Variance tells you how much one quantity spreads. But the interesting data in the world rarely comes one number at a time. You measure a person's height and weight; you track the daily price of several stocks; you record a patient's blood pressure, heart rate and temperature all at once. For a cloud of points like that, you want more than each variable's own spread — you want to know how they move together: do tall people also tend to be heavier? When one stock dips, do the others dip with it?

The covariance matrix is the object that captures the full shape of a multidimensional cloud in one tidy table. Its foundation is the covariance of two variables — the average product of their deviations from their means:

Positive covariance: they tend to rise together. Negative: one rises as the other falls. Zero: no linear relationship. Dividing by the two standard deviations gives the unit-free correlation \rho = \operatorname{cov}(X,Y)/(\sigma_X\sigma_Y) \in [-1, 1].

Packing it into a matrix

For a vector of variables \mathbf{x} = (X_1, \dots, X_n), all the pairwise covariances assemble into the covariance matrix

Read it entry by entry. The diagonal holds each variable's own variance \Sigma_{ii} = \sigma_i^2 — the familiar single-variable spreads. The off-diagonal entries hold the covariances between pairs. And because \operatorname{cov}(X,Y) = \operatorname{cov}(Y,X), the matrix is always symmetric: \Sigma_{ij} = \Sigma_{ji}, so it mirrors across its diagonal. (It is also positive semidefinite — no direction can have negative variance.) Geometrically, \Sigma describes an ellipse of uncertainty: its eigenvectors are the principal axes and its eigenvalues are the variances along them.

A worked 2×2, by hand

Take four paired measurements (X, Y): (2,3),\ (4,7),\ (6,5),\ (8,9). First the means: \mu_X = (2+4+6+8)/4 = 5 and \mu_Y = (3+7+5+9)/4 = 6. Now subtract the means to get the deviations:

The variance of X averages the squared X-deviations: (9+1+1+9)/4 = 5. The variance of Y is the same, 5. The covariance averages the products of the paired deviations:

So the whole matrix is \Sigma = \begin{psmallmatrix}5 & 4\\ 4 & 5\end{psmallmatrix}. Notice it is symmetric — the two 4s are the same covariance written twice. The positive off-diagonal says X and Y are positively correlated: big X went with big Y. Turning that into a correlation strips out the scale:

a strong positive relationship. Correlation is nothing more than covariance renormalised so it always lands in [-1, 1] — a covariance you can compare across datasets.

The ellipse of uncertainty

The ellipse is the one-standard-deviation contour of a 2-D distribution with covariance \Sigma = \begin{psmallmatrix}\sigma_x^2 & \rho\sigma_x\sigma_y\\ \rho\sigma_x\sigma_y & \sigma_y^2\end{psmallmatrix}. Stretch \sigma_x, \sigma_y to change the spread along each axis; turn up the correlation \rho and the ellipse tilts — the variables become entangled, and knowing one tells you about the other.

The bell it shapes

That flat ellipse is a contour line of a hill. Here is the density itself as a rotatable surface — drag it around. Because the covariance is positive, it stretches the density along the diagonal, so the bell leans and a ridge runs corner to corner; an uncorrelated cloud would sit as a round, symmetric bell instead.