Pick up a calculator, put it in radian mode, type any number you like, and press the cos button. Now press it again. And again. And again — twenty, thirty times. Something magical happens: no matter where you started, the display stops moving and freezes at 0.739085\ldots You have just run the oldest and simplest algorithm in numerical analysis by hand. You kept feeding a number back into the same function until it stopped changing, and the value it settled on is a fixed point of cosine — a number that cosine sends back to itself.

This is fixed-point iteration: to solve an equation of the form x = g(x), you start with a guess and simply apply g over and over,

hoping the sequence x_0, x_1, x_2, \ldots homes in on a value the equation is satisfied at. It is the quiet engine hiding inside almost every numerical method — including Newton's method, as we will see at the end. But it comes with a catch that this whole page is about: sometimes the iteration races to the answer, and sometimes it flies apart to infinity — and the difference is governed by a single number, the slope of g at the fixed point.

What is a fixed point?

A fixed point of a function g is an input that the function leaves unmoved:

Geometrically that is nothing more than a point where the graph of y = g(x) crosses the diagonal line y = x. Every crossing of those two curves is a fixed point, and every fixed point is such a crossing — that is the entire picture, and we will lean on it constantly.

Let's see the calculator experiment in code. Iterating x \mapsto \cos x from any start converges to the Dottie number, the unique real solution of x = \cos x:

The display converges on x^\* \approx 0.7390851332, and there \cos x^\* = x^\* exactly to the digits shown. The number even has a name — the Dottie number — and the reason cosine converges while, say, sine's iteration crawls, is the story of this page.

The staircase picture

There is a beautiful way to see an iteration unfold, called a cobweb or staircase diagram. Draw both curves — y = g(x) and the diagonal y = x — and trace the algorithm as a path that bounces between them:

• Start at your guess x_n on the horizontal axis and go vertically to the curve y = g(x). You are now at height g(x_n) — which is the next value x_{n+1}.
• Now go horizontally to the diagonal y = x. This copies that height back onto the horizontal axis, so your new horizontal position is x_{n+1}, ready for the next step.

Repeat, and the path draws a staircase (or a spiral) between the two curves. If the staircase converges into the crossing point, the iteration works; if it marches away from the crossing, the iteration diverges. Whether it closes in or runs off depends entirely on how steep the curve y = g(x) is where it meets the diagonal — a shallow curve pulls the staircase inward, a steep one flings it outward.

Below are two rearrangements of the same equation, plotted against the diagonal. Both cross y = x at the golden ratio \varphi \approx 1.618 — that shared crossing is the fixed point — but one curve is gentle there and one is steep, and that difference decides everything.

When does it converge? The contraction condition

Suppose x^\* is a fixed point and we start nearby. Write the error at step n as e_n = x_n - x^\*. Since x^\* = g(x^\*), the next error is

By the Mean Value Theorem, g(x_n) - g(x^\*) = g'(\xi)\,(x_n - x^\*) for some \xi between them, so

That single line is the heart of the matter. Each step multiplies the error by roughly g'(x^\*). If that multiplier is smaller than 1 in size the error shrinks geometrically and the iteration converges; if it is bigger than 1 the error grows and the iteration diverges. This is a special case of the celebrated Banach fixed-point theorem.

Now the cosine mystery dissolves: for g(x) = \cos x the derivative at the fixed point is g'(x^\*) = -\sin(0.739\ldots) \approx -0.67, comfortably inside [-1, 1], so cosine is a contraction and the calculator always wins.

Rearranging f(x)=0 — and why the arrangement matters

Most problems arrive as "solve f(x) = 0", not "solve x = g(x)". To use fixed-point iteration you must rearrange the equation into the form x = g(x) — and there are always many ways to do it, algebraically identical yet numerically worlds apart. Take the golden-ratio equation

whose positive root is \varphi = \tfrac{1+\sqrt5}{2} \approx 1.618. Two natural rearrangements:

• A. Divide by x and tidy up: x = 1 + \tfrac1x, so g_A(x) = 1 + \tfrac1x. Here g_A'(x) = -1/x^2, and at the root g_A'(\varphi) = -1/\varphi^2 \approx -0.382. Size below 1 — converges.
• B. Move the linear term across: x = x^2 - 1, so g_B(x) = x^2 - 1. Here g_B'(x) = 2x, and at the root g_B'(\varphi) = 2\varphi \approx 3.236. Size above 1 — diverges.

Same equation, same root, opposite fates. Run both side by side from the same neighbourhood and watch column A close in on \varphi while column B bolts:

Column A settles onto 1.618\ldots within a few steps; column B lurches to negatives and thrashes about, never reaching \varphi. The moral is worth tattooing on your arm: whether an iteration converges is a property of the rearrangement g, not of the equation you are solving. A good numerical analyst chooses the arrangement with the flattest g at the root.

The rate is linear — watch the error ratio

The theorem promises linear convergence with rate |g'(x^\*)|: every step the error is multiplied by about that constant. There is a lovely way to confirm this experimentally without knowing the answer to high precision — just watch the ratio of consecutive errors. It should stop wandering and settle onto |g'(x^\*)|. For g_A(x) = 1 + 1/x that limit is 1/\varphi^2 \approx 0.382:

The ratio column marches straight to 0.382 — exactly 1/\varphi^2, the derivative's magnitude at the fixed point. Because the error shrinks by a constant factor each step (not a constant number of digits), linear convergence buys you a fixed number of new correct digits every few steps: with rate 0.382 you gain roughly one decimal digit every 2.4 steps. Steady — but, as the next card shows, we can do far, far better.

Newton's method is fixed-point iteration in disguise

Here is the punchline that ties everything together. Newton's method for solving f(x) = 0,

is exactly a fixed-point iteration x_{n+1} = g(x_n) with the special choice

A root of f is a fixed point of this g (when f(x^\*) = 0, the fraction vanishes and g(x^\*) = x^\*). So why is Newton so much faster than an ordinary iteration? Differentiate g using the quotient rule and something wonderful happens at the root:

because the numerator carries a factor of f(x^\*) = 0. Newton is a fixed-point iteration whose rate |g'(x^\*)| is not just small — it is zero. Linear convergence with rate 0 means the error shrinks faster than any constant factor: in fact e_{n+1} \approx C\,e_n^2, the hallmark quadratic convergence where the number of correct digits roughly doubles every step. Let's confirm it by watching e_{n+1}/e_n^2 settle on a constant (the tell-tale of a quadratic rate) for Newton on f(x) = x^2 - 2, whose iteration is the ancient Babylonian square-root rule g(x) = \tfrac12\!\left(x + \tfrac2x\right):

The e_{n+1}/e_n^2 column holds steady near a constant while the iterate nails \sqrt2 to fifteen digits in about four steps — the same problem an ordinary linear iteration would need dozens of steps for. This is the deep reason Newton earns its keep: it is the fixed-point iteration cleverly engineered so that its convergence rate at the root is driven all the way to zero. Every fixed-point iteration lives on the same spectrum, and its speed is read off one number, |g'(x^\*)| — the closer to zero, the faster you fly.