Truncated SVD
The bluntest regularizer simply throws away the dangerous directions. Since the
trouble lives in the terms with small singular values, the
truncated SVD (TSVD) keeps only the first k components
and discards the rest:
\hat m_k = \sum_{i=1}^{k} \frac{u_i^{\mathsf T}d}{\sigma_i}\,v_i.
Everything past index k — where the singular values are tiny and the
Picard ratio
has turned up — is dropped entirely. No division by near-zero, no noise blow-up.
A hard cutoff vs Tikhonov's soft taper
TSVD is regularization with a brick-wall filter: its filter factors are
f_i = 1 for i \le k and
f_i = 0 beyond. Compare this with
Tikhonov,
whose factors \sigma_i^2/(\sigma_i^2+\alpha^2) taper off
smoothly. They are two points on the same spectrum: both suppress small-singular-value
terms; TSVD does it with a switch, Tikhonov with a dimmer.
The truncation level k is the discrete cousin of
\alpha — the dial that trades resolution against stability. Too small a
k over-smooths (loses detail); too large lets the noise back in.
Switch vs dimmer
The bold step is the TSVD filter — 1 up to the cutoff k, then 0. The
faint curve is the matching Tikhonov filter. Slide k: the wall moves,
keeping more or fewer components. Both keep the strong, trustworthy directions and bin the weak
ones — they differ only in how abruptly they make the cut.
- Keep only the first k SVD components: \hat m_k = \sum_{i \le k}(u_i^{\mathsf T}d/\sigma_i)v_i.
- Filter factors are a hard step (1 then 0) — Tikhonov is the smooth version.
- k plays the role of \alpha: it trades resolution against stability.