Quantifying Uncertainty

The Bayesian payoff is not just a best estimate but a full measure of how much to trust it. With Gaussian likelihood and prior, the posterior is Gaussian, and its spread is the posterior covariance

C_{\text{post}} = \big(G^{\mathsf T} C_D^{-1} G + C_M^{-1}\big)^{-1}.

This single matrix is the uncertainty of the answer. Its diagonal entries are the variances of each recovered parameter — square-root them for error bars. Its off-diagonals say which parameters are correlated, i.e. which trade off against each other in the fit.

Resolution and uncertainty are two sides of one ellipse

The posterior is a multivariate Gaussian, and C_{\text{post}} is its uncertainty ellipse. Well-constrained directions (where the data is informative — large singular values) come out with small variance: the ellipse is thin there. Poorly constrained directions stay wide, inheriting the prior's spread. So the shape of the ellipse reports the resolution direction by direction.

Two ways to shrink it: gather more data (each measurement adds to the precision G^{\mathsf T}C_D^{-1}G) or impose a tighter prior (larger C_M^{-1}) — though a tighter prior shrinks uncertainty by adding bias, the same trade-off once more.

Watch the uncertainty shrink

The ellipse is the posterior covariance for a two-parameter model. With few measurements it is large and tilted (high, correlated uncertainty). Add measurements with the slider: each one adds precision, and the ellipse contracts — fastest along the well-measured direction, slowest along the poorly-measured one, so it does not just shrink but changes shape.