Most equations you meet in the real world cannot be solved with a neat formula. There is no
algebraic rearrangement that solves x = \cos x, or
x^5 - x - 1 = 0, or the integral of
e^{-x^2}. Numerical methods get the answer anyway — not
as an exact expression, but as a number, to as many decimal places as you like — by
iterating: making a guess, improving it, and repeating until it stops changing.
Finding roots
Locating a root. If a continuous function changes sign between
a and b — that is,
f(a) and f(b) have opposite signs — then it
must cross zero somewhere between them. That single observation pins a root to an interval, which you
can then shrink.
Fixed-point iteration. Rearrange f(x)=0 into the form
x = g(x), then just keep feeding the output back in:
x_{n+1} = g(x_n). If the rearrangement is a good one (technically
|g'(x)| < 1 near the root) the values home in on the solution.
Newton–Raphson. The fastest classic method uses the
derivative.
From a guess x_n, follow the tangent line down to where it
crosses the axis, and use that crossing as your next guess:
x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}.
Each step roughly doubles the number of correct digits — astonishingly fast. Here
it computes \sqrt{2} by solving x^2 - 2 = 0;
watch the digits lock in. Press Run:
// Solve f(x) = 0 by Newton-Raphson: x = x - f(x)/f'(x).
function newton(
f: (x: number) => number,
df: (x: number) => number,
x0: number,
steps: number,
): number {
let x = x0;
for (let i = 0; i < steps; i++) {
x = x - f(x) / df(x);
console.log("step " + (i + 1) + ": x = " + x);
}
return x;
}
// f(x) = x^2 - 2 has root sqrt(2); f'(x) = 2x.
newton((x) => x * x - 2, (x) => 2 * x, 1, 6);
console.log("sqrt(2) exact = 1.4142135623730951");
From a rough start of 1, it nails \sqrt{2} to
full precision in just a handful of steps. This is, essentially, how your calculator finds square
roots.
Numerical integration
The same "approximate, then refine" idea handles integrals that have no elementary antiderivative.
The trapezium rule slices the area under a curve into thin strips and approximates
each by a trapezium — straight tops instead of the true curve — then adds them up. With
n strips of width h:
\int_a^b f(x)\,dx \approx \tfrac{h}{2}\Big[f(x_0) + 2f(x_1) + \cdots + 2f(x_{n-1}) + f(x_n)\Big].
The more strips, the finer the approximation — refine until the answer stops moving.
- A sign change of a continuous function brackets a root.
- Newton–Raphson:
x_{n+1} = x_n - f(x_n)/f'(x_n) — very fast near a root.
- The trapezium rule approximates a definite integral by summing thin
trapezia; more strips → more accuracy.
When you press \sqrt{\ }, \sin, or
\ln, your calculator does not consult a giant lookup table. It
computes the answer on the spot by numerical methods — Newton-style iterations, or
rapidly-converging series, refined until every displayed digit is correct. Every "exact-looking"
value on the screen is really a numerical approximation that has converged past the last digit you
can see.
-
Newton–Raphson fails if f'(x_n) = 0 (a flat tangent
never reaches the axis), and a poor starting guess can make it diverge or
oscillate instead of converging.
-
Fixed-point iteration only converges when |g'| < 1 near the root —
a bad rearrangement will fly apart. And a sign-change search misses roots where the curve
touches the axis without crossing it.