Why Inverse Problems Are Hard

Jacques Hadamard called a problem well-posed if it has the three properties any sensible model of nature should: a solution exists, it is unique, and it depends continuously on the data (small data changes → small solution changes). Forward problems are usually well-posed. Inverse problems usually are not — they are ill-posed, and the way they fail is the whole story.

The three failures, in matrix terms

For the linear problem d = Gm, each Hadamard condition has a crisp linear-algebra meaning:

Existence fails when d lies outside the column space of G — no model reproduces the (noisy) data exactly. Fix: accept a best fit instead of an exact one.
Uniqueness fails when G has a non-trivial null space: any m_0 with Gm_0 = 0 can be added to a solution and the data cannot tell. Fix: pick a preferred solution (minimum norm, or a prior).
Stability fails when G has tiny singular values: inverting divides by them, so noise is amplified enormously. Fix: regularization.

The first two can be repaired by the generalized inverse. The third — instability — is the deep difficulty, and the reason this course spends so long on conditioning and regularization. (It is the same instability that made the backward heat equation hopeless.)

Existence — a solution exists (fails if d \notin column space of G).
Uniqueness — only one solution (fails if G has a null space).
Stability — continuous dependence on data (fails when G has tiny singular values).
Inverse problems are typically ill-posed; instability is the hardest failure to cure.