The Multivariate Gaussian
The bell curve generalises to many dimensions. A multivariate Gaussian is
pinned down by two objects: a mean vector \boldsymbol\mu
(where it is centred) and a
covariance matrix
\Sigma (its shape and spread). Its density is
p(\mathbf{x}) = \frac{1}{\sqrt{(2\pi)^n \det\Sigma}}\,\exp\!\Big(-\tfrac12 (\mathbf{x} - \boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\mathbf{x} - \boldsymbol\mu)\Big).
The exponent is the heart of it. The quadratic form
(\mathbf{x} - \boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\mathbf{x} - \boldsymbol\mu)
is the squared Mahalanobis distance — distance measured in units of standard
deviation, stretched by the covariance. It reappears verbatim as the data-misfit term in the
Bayesian inverse problem.
Contours are ellipses
The density is constant where the quadratic form is constant, so the level sets are
ellipses — concentric, centred at \boldsymbol\mu, with axes
and widths set by the eigenvectors and eigenvalues of \Sigma. A
diagonal \Sigma gives axis-aligned ellipses; correlation tilts them.
A special fact peculiar to Gaussians: zero covariance implies independence. For
a general distribution uncorrelated does not mean independent, but for jointly Gaussian variables
a diagonal \Sigma means the components are genuinely independent.
Nested probability contours
The nested ellipses are equal-density contours of a centred 2-D Gaussian. Adjust
\sigma_x, \sigma_y and the correlation \rho:
the inner ellipse is the most probable region, and each outer ring is less likely. The whole
family rotates and stretches together because they all share the one matrix
\Sigma.
- Fully described by mean vector \boldsymbol\mu and covariance \Sigma.
- The exponent is the Mahalanobis distance (\mathbf{x}-\boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu); level sets are ellipses.
- For Gaussians, zero covariance ⇒ independence.