Solving PDEs Numerically
Touch one end of a metal rod to a flame and, a little while later, the far end grows warm. Drop a
bead of ink into a glass of still water and watch the dark blot blur outward until the whole glass is
faintly grey. In both cases something — heat, dye — spreads: it flows from where
there is a lot of it to where there is little, smoothing every bump as it goes. The law behind this
is the heat equation (or diffusion equation), and it is a
partial differential equation (PDE) — an equation that ties together how a quantity
changes in space and how it changes in time at the same moment:
\frac{\partial u}{\partial t} = \alpha\,\frac{\partial^2 u}{\partial x^2}.
Here u(x,t) is the temperature at position x
along the rod at time t, and \alpha is the
diffusivity — how eagerly the material shuffles heat around. In words the equation
says: a point heats up (or cools) at a rate set by how curved the temperature profile is right
there. A sharp peak (strongly curved, \partial^2 u/\partial x^2 < 0)
collapses; a valley fills in. Left alone, every wrinkle irons itself flat.
The trouble is that only a handful of PDEs can be solved with pen and paper, and only for the tidiest
shapes and boundaries. The rod in your hand, with its lumpy initial temperature and its clamped ends,
is not one of them. So we do what all of computational physics does: we put the field on a
grid, replace the derivatives with differences, and march the whole thing forward one small step at a
time. This page builds that recipe — the finite-difference method — from
scratch, runs it live in your browser, and then confronts the one lesson that trips up every
newcomer: do it carelessly and the numbers do not just get inaccurate, they explode.
Step 1 — put the field on a grid
A function u(x,t) holds a value at infinitely many points; a computer can
only store finitely many numbers. So we sample. Chop the rod into equal slices of
width \Delta x and chop time into equal ticks of length
\Delta t. We only ever track the temperature at the crossing points of
this lattice, and we give each one a pair of indices:
u_i^{\,n} \;\approx\; u\bigl(x_i,\, t_n\bigr), \qquad x_i = i\,\Delta x, \qquad t_n = n\,\Delta t.
Read it aloud: u_i^{\,n} is "the temperature at the
i-th point in space, at the n-th moment in
time." The subscript i walks along the rod; the
superscript n counts the clock ticks (it is an index, not a
power). A whole row u_0^n, u_1^n, \dots, u_M^n is a single snapshot of the
rod at one instant. Our job is to turn each snapshot into the next one.
Step 2 — turn derivatives into differences
A derivative is a limit of a ratio of small differences. On a grid we simply stop taking the
limit and keep the ratio. That is the whole trick. For the time derivative we look one tick
ahead — a forward difference:
\frac{\partial u}{\partial t}\bigg|_i^{\,n} \;\approx\; \frac{u_i^{\,n+1} - u_i^{\,n}}{\Delta t}.
For the second derivative in space we compare a point with its two neighbours. If the point sits
below the average of its neighbours the profile is cupped upward (a valley); if above, it is a peak.
The central second difference measures exactly that:
\frac{\partial^2 u}{\partial x^2}\bigg|_i^{\,n} \;\approx\; \frac{u_{i+1}^{\,n} - 2\,u_i^{\,n} + u_{i-1}^{\,n}}{\Delta x^2}.
The pattern (\,\text{left} - 2\times\text{middle} + \text{right}\,) is the
discrete curvature: it is zero for a straight line, negative at a hump, positive in a dip. It is worth
knowing by heart — it is the workhorse of the whole subject.
Slide the first derivative onto half-way points. The slope just to the right of
i is (u_{i+1}-u_i)/\Delta x; the slope just to
the left is (u_i-u_{i-1})/\Delta x. The second derivative is how
fast the slope itself changes, so subtract the two slopes and divide by
\Delta x again:
\frac{1}{\Delta x}\!\left(\frac{u_{i+1}-u_i}{\Delta x} - \frac{u_i-u_{i-1}}{\Delta x}\right) = \frac{u_{i+1} - 2u_i + u_{i-1}}{\Delta x^2}.
A Taylor-series check shows the error is proportional to \Delta x^2 — a
second-order accurate stencil, which is why it is so popular: halve the spacing and
the spatial error drops fourfold.
Step 3 — the FTCS update rule
Now substitute both approximations into the heat equation
\partial u/\partial t = \alpha\,\partial^2 u/\partial x^2 and solve for the
one thing we do not yet know — the temperature at the next tick,
u_i^{\,n+1}:
\frac{u_i^{\,n+1} - u_i^{\,n}}{\Delta t} = \alpha\,\frac{u_{i+1}^{\,n} - 2u_i^{\,n} + u_{i-1}^{\,n}}{\Delta x^2}.
Multiply through by \Delta t, gather the constant into a single
dimensionless number
r \;=\; \frac{\alpha\,\Delta t}{\Delta x^2},
and out drops the explicit FTCS update — "Forward-Time, Central-Space":
-
The update.
u_i^{\,n+1} = u_i^{\,n} + r\bigl(u_{i+1}^{\,n} - 2u_i^{\,n} + u_{i-1}^{\,n}\bigr), \qquad r = \frac{\alpha\,\Delta t}{\Delta x^2}.
-
It is a weighted average. Regroup it as
u_i^{\,n+1} = (1-2r)\,u_i^{\,n} + r\,u_{i+1}^{\,n} + r\,u_{i-1}^{\,n}:
each new value is a blend of a cell and its two neighbours. At
r = \tfrac12 the middle weight vanishes and the cell becomes the plain
average of its neighbours — pure smoothing.
-
It is explicit. Every term on the right is known from the current row, so we get
the whole next row by direct arithmetic — no equations to solve. Cheap per step, but, as we will
see, only conditionally stable.
And that is the entire algorithm: from one snapshot compute the next by sweeping this formula across
the interior points, then repeat. We only need two more ingredients to actually run it — where to
start, and what to do at the ends.
Starting values and the ends of the rod
A PDE on its own has infinitely many solutions; to pick out our rod we pin down two things.
-
Initial condition — the temperature everywhere at
t = 0, i.e. the whole starting row u_i^{\,0}.
Maybe a hot spike in the middle, maybe a step (left half hot, right half cold).
-
Boundary conditions — what happens at the two ends,
every tick. The simplest are Dirichlet conditions: hold the end
temperatures fixed (say both ends clamped in ice at u = 0). In code we
simply never update u_0 and u_M — the sweep
runs only over the interior points i = 1,\dots,M-1.
With those in hand the loop is trivial to write. Here it is — a hot spike planted in the centre of an
ice-clamped rod, marched forward with a stable
r = 0.4. Each printed row is a snapshot: the bars are the temperature of
each cell, so you can literally watch the spike slump and spread outward while its peak sinks. Press
Run ▶, then change a number and run it again.
const alpha = 1.0; // diffusivity
const dx = 0.1; // grid spacing
const dt = 0.004; // time step
const r = alpha * dt / (dx * dx); // = 0.4 -> stable, since r <= 0.5
console.log("r = alpha*dt/dx^2 = " + r.toFixed(2) +
(r <= 0.5 ? " (stable)" : " (UNSTABLE!)"));
const N = 11; // 11 grid points, i = 0..10
let u: number[] = new Array(N).fill(0);
u[5] = 1; // a hot spike in the middle
function draw(n: number, u: number[]): void {
const bars = u
.map((v) => "#".repeat(Math.max(0, Math.round(v * 8))).padEnd(8, "."))
.join(" ");
console.log("n=" + String(n).padStart(2, " ") + " | " + bars);
}
draw(0, u);
for (let n = 1; n <= 60; n++) {
const next = u.slice();
for (let i = 1; i < N - 1; i++) { // interior only: ends stay clamped at 0
next[i] = u[i] + r * (u[i + 1] - 2 * u[i] + u[i - 1]);
}
u = next;
if (n % 15 === 0) draw(n, u);
}
console.log("The peak sinks and the heat oozes outward: diffusion at work.");
The catch: stability
You might reasonably expect that a smaller time step \Delta t always makes
the answer better, and that a bigger one just makes it a bit rougher. For this explicit scheme that is
false, and the failure is spectacular. Push \Delta t too
far and the solution does not degrade gently — it erupts into wild, alternating-sign oscillations that
double in size every few steps until the numbers overflow. There is a hard line:
-
The explicit heat-equation update is stable only if
r = \frac{\alpha\,\Delta t}{\Delta x^2} \;\le\; \frac{1}{2}, \qquad\text{equivalently}\qquad \Delta t \;\le\; \frac{\Delta x^2}{2\alpha}.
-
Cross that line and errors are amplified every step; the numerical solution blows
up even though the true physical solution is perfectly smooth and bounded.
Where does the magic \tfrac12 come from? Here is the intuition (a
von Neumann stability argument, stripped to its bones). Any error on the grid can be
built from wavy pieces. Feed the single wiggliest wave the grid can hold — one that
flips sign at every point, \dots, +1, -1, +1, -1, \dots — into
the update. For that pattern u_{i+1} = u_{i-1} = -u_i, so
u_i^{\,n+1} = u_i^{\,n} + r(-u_i - 2u_i - u_i) = (1 - 4r)\,u_i^{\,n}.
Each step just multiplies that wave by the amplification factor
g = 1 - 4r. The wave stays under control only if
|g| \le 1, i.e. -1 \le 1 - 4r \le 1, which
rearranges to 0 \le r \le \tfrac12. The moment
r > \tfrac12 we get g < -1: every step the
jagged wave grows and flips sign — precisely the exploding zig-zag you are about to see. This
is a member of the famous CFL family of conditions (Courant–Friedrichs–Lewy) that
govern nearly every explicit scheme in computational physics: the time step is chained to the grid
spacing, and you cannot cheat it.
Watch it happen. The next block is the same solver as before — same rod, same spike — but
with \Delta t nudged up so that r = 0.8 > \tfrac12.
We print the raw numbers so you can see the sign flip and the magnitude run away:
const alpha = 1.0;
const dx = 0.1;
const dt = 0.008; // just twice the previous step...
const r = alpha * dt / (dx * dx); // = 0.8 -> r > 1/2, UNSTABLE
console.log("r = " + r.toFixed(2) +
(r <= 0.5 ? " (stable)" : " (UNSTABLE, r > 1/2)"));
console.log("amplification of the jagged mode g = 1 - 4r = " + (1 - 4 * r).toFixed(2) +
" (|g| > 1 -> grows!)");
console.log("");
const N = 11;
let u: number[] = new Array(N).fill(0);
u[5] = 1;
function show(n: number, u: number[]): void {
console.log("n=" + String(n).padStart(2, " ") + " " +
u.map((v) => v.toFixed(2).padStart(9, " ")).join(""));
}
show(0, u);
for (let n = 1; n <= 24; n++) {
const next = u.slice();
for (let i = 1; i < N - 1; i++) {
next[i] = u[i] + r * (u[i + 1] - 2 * u[i] + u[i - 1]);
}
u = next;
if (n <= 4 || n % 4 === 0) show(n, u);
}
console.log("\nAlternating signs, growing without bound: an instability, not a bug.");
Pinning down the threshold
The r = \tfrac12 boundary is sharp — worth seeing directly rather than
taking on faith. This last program starts every run from the worst possible initial
condition (the sign-flipping zig-zag, the fastest-growing mode) and sweeps a range of time steps. For
each it marches 200 steps and reports the peak size. Below the line the peak stays near
1; a hair above it, the numbers detonate:
const alpha = 1.0;
const dx = 0.1;
const N = 21;
for (const dt of [0.002, 0.004, 0.005, 0.006, 0.008]) {
const r = alpha * dt / (dx * dx);
// worst case: a wave that flips sign at every grid point
let u: number[] = Array.from({ length: N }, (_, i) => (i % 2 === 0 ? 1 : -1));
u[0] = 0;
u[N - 1] = 0;
for (let n = 0; n < 200; n++) {
const next = u.slice();
for (let i = 1; i < N - 1; i++) {
next[i] = u[i] + r * (u[i + 1] - 2 * u[i] + u[i - 1]);
}
u = next;
}
const peak = Math.max(...u.map((v) => Math.abs(v)));
console.log("r = " + r.toFixed(2) +
" peak after 200 steps = " + peak.toExponential(2) +
(r <= 0.5 ? " bounded" : " BLEW UP"));
}
console.log("\nThe knife-edge sits exactly at r = 0.5.");
Everything with r \le 0.5 stays tame; the very first value above it
(r = 0.6) has already grown by dozens of orders of magnitude. That is the
stability condition, made concrete.
Watching heat diffuse
Run the stable scheme long enough on a concentrated blob of heat and the profile relaxes into a
smooth, ever-widening bell that flattens as it spreads — the exact quantity of heat is conserved, it
just gets shared out. The graph below shows that idealised spreading: the temperature profile
u(x) at four successive times, narrow-and-tall giving way to
broad-and-low.
Notice how the tallest, most sharply curved peak changes fastest: strong curvature means a large
\partial^2 u/\partial x^2, and the heat equation turns curvature directly
into rate of change. As the profile flattens, its curvature drops, and the spreading slows — which is
why the last two snapshots look far more alike than the first two.
The obvious way to get a more accurate answer in space is to shrink \Delta x
— use more grid points. But look again at the stability limit:
\Delta t \;\le\; \frac{\Delta x^2}{2\alpha}.
The time step is capped by \Delta x^{2}, not
\Delta x. Halve \Delta x and the largest
allowed \Delta t falls by a factor of four. Refine the grid by
10\times and you must take 100\times as many time
steps — and each step now sweeps 10\times more points, so the total work
balloons by roughly 1000\times. This quadratic tyranny is the explicit
scheme's Achilles' heel, and the usual reason people reach for the implicit methods in the next box.
A second, related trap: when a run first blows up, the instinct is to hunt for a coding bug. Before
you do, check r. If r > \tfrac12
the "bug" is the mathematics doing exactly what it must — the fix is a smaller
\Delta t, not a smaller typo.
Yes — and the trick is to evaluate the right-hand side at the new time level instead of the
old one. That gives the implicit (backward-Euler) scheme,
u_i^{\,n+1} - r\bigl(u_{i+1}^{\,n+1} - 2u_i^{\,n+1} + u_{i-1}^{\,n+1}\bigr) = u_i^{\,n},
and its symmetric cousin, Crank–Nicolson (an average of the explicit and implicit
updates), which is second-order accurate in time as well. Both are
unconditionally stable: you may take as large a \Delta t
as accuracy allows and the solution never explodes.
Nothing is free, though. The unknowns u^{\,n+1} now appear on
both sides and are tangled together, so each step is no longer plain arithmetic — it requires
solving a linear system for the whole new row at once. Happily that system is
tridiagonal (each equation touches only a cell and its two neighbours), so it falls to a fast
specialised solver. This is the classic trade of computational physics: an explicit method is cheap
per step but shackled by stability; an implicit method costs a linear solve per step but takes giant
strides. Which wins depends on the problem — but knowing both, and knowing why the explicit
one is chained to r \le \tfrac12, is the whole game.