Numerical Integration of ODEs
Here is an uncomfortable secret that textbooks tend to whisper: most equations of motion have
no formula for the answer. Newton hands you F = ma, you rearrange
it into a differential equation for where things go — and then, for almost anything more interesting
than a ball thrown in a vacuum, you discover there is no tidy closed-form
x(t) to write down. Three planets tugging on each other? No formula. A
pendulum swung hard? No elementary formula. Air resistance that grows with speed, a rocket that loses
mass as it burns, a charged particle spiralling through a messy magnetic field? No, no, and no.
So how does a spacecraft find Jupiter, or a game make a thousand asteroids drift convincingly, or a
climate model run for a century? The trick is to stop asking for the whole future at once and instead
take one tiny step at a time. If you know exactly where something is and how fast
it's moving right now, and you know the forces on it, then you know which way it's about to
head. Nudge it forward by a sliver of time \Delta t, recompute the forces
at the new spot, and step again. Repeat a few thousand times and a trajectory unrolls in front of you.
That is numerical integration of an ordinary differential equation (ODE): trading one
impossible act of algebra for a great many easy acts of arithmetic. This page is about how to take
those steps — and, just as importantly, how not to.
Step one: turn F = ma into a first-order system
Newton's law is a second-order equation — it involves an acceleration, which is a
second derivative of position:
m\,\frac{d^2 x}{dt^2} = F(x, v, t) \quad\Longleftrightarrow\quad \frac{d^2 x}{dt^2} = \frac{F}{m} = a.
Every stepping method we'll meet is built to march a first-order equation forward — something
of the shape \dfrac{dy}{dt} = f(y, t), "the rate of change of the state is a
known function of the state." The universal move that unlocks everything is to promote velocity
to a state variable of its own. Instead of one second-order equation for
x, keep track of the pair (x, v) and write
two first-order equations:
-
The state. Bundle position and velocity into one vector,
\mathbf{y} = (x, v). Knowing \mathbf{y} at an
instant is knowing everything you need to predict the next instant.
-
Position changes at the rate of velocity:
\dfrac{dx}{dt} = v.
-
Velocity changes at the rate of acceleration:
\dfrac{dv}{dt} = a = \dfrac{F(x, v, t)}{m}.
That's it — a second-order law became a first-order system in twice as many variables. In three
dimensions the state is six numbers (x, y, z, v_x, v_y, v_z); for
N planets it's 6N. The machinery below never
needs to know it came from Newton; it just marches the state vector
\mathbf{y} forward using its rate of change
f(\mathbf{y}, t). Reduce first, then step.
Euler's method: the most honest, most naive step
The simplest possible idea is due to Leonhard Euler in the 1760s and reads almost like common sense:
assume the rate of change holds steady across one small step. If you're moving at
velocity v_n right now, then over the next \Delta t
you'll travel about v_n\,\Delta t; if your acceleration is
a_n, your velocity changes by about a_n\,\Delta t.
Turn the crank on the whole state:
x_{n+1} = x_n + v_n\,\Delta t, \qquad v_{n+1} = v_n + a_n\,\Delta t, \qquad a_n = \frac{F(x_n, v_n, t_n)}{m}.
Where does this come from, precisely? From chopping the Taylor series off after one
term. The exact future position is
x(t_n + \Delta t) = x_n + \underbrace{\dot x_n\,\Delta t}_{\text{Euler keeps this}} + \underbrace{\tfrac{1}{2}\ddot x_n\,\Delta t^2 + \cdots}_{\text{Euler throws this away}}.
Euler keeps the straight-line term and discards everything from
\Delta t^2 onward. So the error made in a single step — the
local truncation error — is of order \Delta t^2. But to
integrate over a fixed span of time T you need
T/\Delta t steps, and the little errors accumulate. Multiply an error of
size \Delta t^2 by a number of steps proportional to
1/\Delta t and you get a global error of order
\Delta t. Euler is a first-order method: halve the step
size and you (roughly) halve the final error. That is a painfully slow bargain — to get one more
decimal digit of accuracy you must take ten times as many steps.
Watch Euler in action — and watch it cheat
Let's integrate the friendliest system in physics: a mass on a spring, the simple harmonic
oscillator. The force is F = -kx, so the acceleration is
a = -\tfrac{k}{m}x — always pulling back toward the origin. Take
k = m = 1, release the mass from x = 1 at rest,
and the true motion is a perfect cosine that swings forever between
-1 and +1. Its energy
E = \tfrac12 m v^2 + \tfrac12 k x^2 should sit rock-steady at
0.5 for all time. Run Euler and read the last column carefully:
// Explicit (forward) Euler on a spring: a = -(k/m) x
const k = 1, m = 1;
const dt = 0.1;
let x = 1, v = 0; // released from rest at x = 1
const energy = () => 0.5 * m * v * v + 0.5 * k * x * x;
console.log("Euler — energy SHOULD stay at 0.500");
console.log(" t x v E");
for (let n = 0; n <= 60; n++) {
if (n % 10 === 0) {
console.log(
`${(n * dt).toFixed(1).padStart(4)} ` +
`${x.toFixed(3).padStart(7)} ` +
`${v.toFixed(3).padStart(7)} ` +
`${energy().toFixed(3).padStart(6)}`,
);
}
const a = -(k / m) * x; // acceleration from the CURRENT position
const xNext = x + v * dt; // both updates use the OLD values
const vNext = v + a * dt;
x = xNext;
v = vNext;
}
The energy climbs, step after step — the oscillation slowly spirals outward instead of
tracing a closed loop. Nothing is wrong with the code; this is Euler being Euler. Each straight-line
step overshoots the true curved path just slightly outward, and for this system those overshoots always
add energy rather than remove it. Left running for a few hundred steps the amplitude balloons
absurdly. A method that invents energy from nothing is a serious problem if you were hoping to
model, say, a stable planetary orbit.
Seeing the drift: energy over time
The plot below runs both Euler and the fancier RK4 method (coming up next) on that same unit spring and
tracks the energy as time marches on. The true energy is the flat line at
0.5. Watch Euler peel away from it while RK4 clings to it.
Same equation, same step size — wildly different faithfulness. The gap is not about doing more steps;
it's about being smarter within each step. That is exactly what Runge and Kutta figured out.
Runge–Kutta 4: sample the slope, then average it wisely
Euler's flaw is that it commits to the slope measured at the start of the step and rides it
blindly all the way across, even though the true slope is changing the whole time. The
fourth-order Runge–Kutta method (RK4), published around 1900, fixes this by taking
four trial peeks at the slope inside a single step and blending them into one superior estimate. Writing
the state's rate of change as f(\mathbf{y}, t), one RK4 step from
\mathbf{y}_n is:
\begin{aligned}
k_1 &= f\!\left(\mathbf{y}_n,\; t_n\right) &&\text{slope at the start} \\
k_2 &= f\!\left(\mathbf{y}_n + \tfrac{\Delta t}{2}k_1,\; t_n + \tfrac{\Delta t}{2}\right) &&\text{slope at the midpoint, using } k_1 \\
k_3 &= f\!\left(\mathbf{y}_n + \tfrac{\Delta t}{2}k_2,\; t_n + \tfrac{\Delta t}{2}\right) &&\text{midpoint again, using the better } k_2 \\
k_4 &= f\!\left(\mathbf{y}_n + \Delta t\,k_3,\; t_n + \Delta t\right) &&\text{slope at the end, using } k_3
\end{aligned}
Then step forward using a weighted average of the four, leaning hardest on the two
midpoint estimates:
\mathbf{y}_{n+1} = \mathbf{y}_n + \frac{\Delta t}{6}\Bigl(k_1 + 2k_2 + 2k_3 + k_4\Bigr).
The weights \tfrac16, \tfrac13, \tfrac13, \tfrac16 are not plucked from the
air: they are precisely the numbers that make the method's Taylor expansion agree with the true
solution all the way out to the \Delta t^4 term. That buys an enormous prize.
RK4's local error is O(\Delta t^5) and its global error is
O(\Delta t^4). Halve the step and the error drops by a factor of
2^4 = 16, not a measly factor of 2. RK4 costs four force evaluations per step
instead of Euler's one, but for any accuracy you actually care about it wins by a landslide — which is
why it is the reliable workhorse behind countless simulators.
// RK4 on the SAME spring. State is the tuple y = [x, v]; f(y) = [v, -(k/m) x].
const k = 1, m = 1, dt = 0.1;
type State = [number, number];
const deriv = ([x, v]: State): State => [v, -(k / m) * x];
const step = (y: State, s: State, h: number): State => [y[0] + h * s[0], y[1] + h * s[1]];
const energy = ([x, v]: State) => 0.5 * m * v * v + 0.5 * k * x * x;
let y: State = [1, 0];
console.log("RK4 — energy stays glued to 0.500");
console.log(" t x v E");
for (let n = 0; n <= 60; n++) {
if (n % 10 === 0) {
console.log(
`${(n * dt).toFixed(1).padStart(4)} ` +
`${y[0].toFixed(3).padStart(7)} ` +
`${y[1].toFixed(3).padStart(7)} ` +
`${energy(y).toFixed(3).padStart(6)}`,
);
}
const k1 = deriv(y);
const k2 = deriv(step(y, k1, dt / 2));
const k3 = deriv(step(y, k2, dt / 2));
const k4 = deriv(step(y, k3, dt));
y = [
y[0] + (dt / 6) * (k1[0] + 2 * k2[0] + 2 * k3[0] + k4[0]),
y[1] + (dt / 6) * (k1[1] + 2 * k2[1] + 2 * k3[1] + k4[1]),
];
}
Read the energy column: it holds at 0.500 to three decimals across the whole
run. Four cheap peeks per step, and the spurious drift essentially vanishes.
A real trajectory: a planet in orbit
Now for something with no closed-form shortcut in general — a body falling around a star under gravity.
The acceleration points toward the centre and weakens with the square of the distance:
\mathbf{a} = -\,\frac{GM}{r^2}\,\hat{\mathbf{r}} = -\,\frac{GM}{r^3}\,\mathbf{r}, \qquad r = |\mathbf{r}|.
The state is now four numbers (x, y, v_x, v_y). We'll step it with
symplectic Euler — a one-line twist on plain Euler that updates the velocity
first and then advances the position using that new velocity. That tiny reordering
makes the method conserve energy over the long haul (up to a small bounded wobble), which is exactly
what you want for an orbit that should stay closed. Set GM = 1 and launch
the body from radius 1 with just the right sideways speed for a circle; the
energy should hover near E = \tfrac12 v^2 - \tfrac{GM}{r} = -0.5.
// A 2-D gravitational orbit, stepped with SYMPLECTIC Euler (velocity first, then position).
const GM = 1, dt = 0.05;
let x = 1, y = 0, vx = 0, vy = 1; // circular orbit: v = sqrt(GM/r) = 1 at r = 1
const energy = () => {
const r = Math.sqrt(x * x + y * y);
return 0.5 * (vx * vx + vy * vy) - GM / r; // should sit near -0.500
};
console.log("Symplectic-Euler orbit — bound, energy near -0.500, r near 1");
console.log(" t x y r E");
const steps = 126; // ~ one full revolution (period 2π)
for (let n = 0; n <= steps; n++) {
const r = Math.sqrt(x * x + y * y);
if (n % 21 === 0) {
console.log(
`${(n * dt).toFixed(2).padStart(5)} ` +
`${x.toFixed(3).padStart(7)} ` +
`${y.toFixed(3).padStart(7)} ` +
`${r.toFixed(3).padStart(7)} ` +
`${energy().toFixed(3).padStart(7)}`,
);
}
const ax = -GM * x / (r * r * r);
const ay = -GM * y / (r * r * r);
vx += ax * dt; // 1) velocity uses the current acceleration...
vy += ay * dt;
x += vx * dt; // 2) ...position uses the freshly-updated velocity
y += vy * dt;
}
The radius stays near 1 and after roughly one period
(t \approx 2\pi \approx 6.28) the body arrives back near its launch point:
a genuine bound orbit, computed from nothing but the force law and a few hundred additions. Try plain
forward Euler on this same setup and the planet spirals slowly out into deep space — a fake, and a
catastrophic one for anything meant to run for millions of steps.
The trap: "smaller \Delta t always means a better answer."
It feels obvious — tinier steps track the curve more faithfully, so shrink
\Delta t as far as you can and the error must keep falling. It does, for a
while. But every step also does floating-point arithmetic, and each operation carries a whisper of
round-off error. Halve \Delta t and you double the number of
steps, so you double the round-off you accumulate. Past a certain sweet-spot step size, the truncation
error you're driving down is overtaken by round-off error you're piling up, and the total error
starts getting worse as you shrink the step — all while the computation takes far longer. There
is an optimum \Delta t, not a limit of "as small as possible."
A companion confusion worth clearing up: local error is not global error. Euler's
per-step (local) error is O(\Delta t^2), which sounds respectable —
but because the number of steps grows like 1/\Delta t, the errors accumulate
into a global error of only O(\Delta t). Always quote the global
order (Euler 1, RK4 4) when you compare methods; the per-step figure flatters everyone by one power.
Because accuracy over one step is not the same as good behaviour over a billion steps.
RK4 is beautifully accurate locally, yet run for astronomically many steps it still slowly bleeds or
gains energy, and a planetary system left to simmer for a simulated age will quietly drift off its true
path. What long-run simulations really want is a method that conserves energy on average
forever, even if any single step is a touch less accurate. Those are the symplectic
integrators — they respect the deep geometric structure of Newtonian mechanics (they preserve
area in the position–velocity phase space), so their energy error stays bounded and oscillating instead
of marching off to infinity. The symplectic Euler we used for the orbit is the baby of the family; its
famous, only-slightly-bigger sibling is the leapfrog (a.k.a. Verlet) integrator, the
quiet default powering everything from galaxy formation codes to the molecular-dynamics engines that fold
proteins. It updates position and velocity in an interleaved, time-symmetric "leapfrog" — and it is
second-order accurate for the price of a single force evaluation per step. A story for its own page.