Numerical Integration of ODEs

Here is an uncomfortable secret that textbooks tend to whisper: most equations of motion have no formula for the answer. Newton hands you F = ma, you rearrange it into a differential equation for where things go — and then, for almost anything more interesting than a ball thrown in a vacuum, you discover there is no tidy closed-form x(t) to write down. Three planets tugging on each other? No formula. A pendulum swung hard? No elementary formula. Air resistance that grows with speed, a rocket that loses mass as it burns, a charged particle spiralling through a messy magnetic field? No, no, and no.

So how does a spacecraft find Jupiter, or a game make a thousand asteroids drift convincingly, or a climate model run for a century? The trick is to stop asking for the whole future at once and instead take one tiny step at a time. If you know exactly where something is and how fast it's moving right now, and you know the forces on it, then you know which way it's about to head. Nudge it forward by a sliver of time \Delta t, recompute the forces at the new spot, and step again. Repeat a few thousand times and a trajectory unrolls in front of you. That is numerical integration of an ordinary differential equation (ODE): trading one impossible act of algebra for a great many easy acts of arithmetic. This page is about how to take those steps — and, just as importantly, how not to.

Step one: turn F = ma into a first-order system

Newton's law is a second-order equation — it involves an acceleration, which is a second derivative of position:

m\,\frac{d^2 x}{dt^2} = F(x, v, t) \quad\Longleftrightarrow\quad \frac{d^2 x}{dt^2} = \frac{F}{m} = a.

Every stepping method we'll meet is built to march a first-order equation forward — something of the shape \dfrac{dy}{dt} = f(y, t), "the rate of change of the state is a known function of the state." The universal move that unlocks everything is to promote velocity to a state variable of its own. Instead of one second-order equation for x, keep track of the pair (x, v) and write two first-order equations:

That's it — a second-order law became a first-order system in twice as many variables. In three dimensions the state is six numbers (x, y, z, v_x, v_y, v_z); for N planets it's 6N. The machinery below never needs to know it came from Newton; it just marches the state vector \mathbf{y} forward using its rate of change f(\mathbf{y}, t). Reduce first, then step.

Euler's method: the most honest, most naive step

The simplest possible idea is due to Leonhard Euler in the 1760s and reads almost like common sense: assume the rate of change holds steady across one small step. If you're moving at velocity v_n right now, then over the next \Delta t you'll travel about v_n\,\Delta t; if your acceleration is a_n, your velocity changes by about a_n\,\Delta t. Turn the crank on the whole state:

x_{n+1} = x_n + v_n\,\Delta t, \qquad v_{n+1} = v_n + a_n\,\Delta t, \qquad a_n = \frac{F(x_n, v_n, t_n)}{m}.

Where does this come from, precisely? From chopping the Taylor series off after one term. The exact future position is

x(t_n + \Delta t) = x_n + \underbrace{\dot x_n\,\Delta t}_{\text{Euler keeps this}} + \underbrace{\tfrac{1}{2}\ddot x_n\,\Delta t^2 + \cdots}_{\text{Euler throws this away}}.

Euler keeps the straight-line term and discards everything from \Delta t^2 onward. So the error made in a single step — the local truncation error — is of order \Delta t^2. But to integrate over a fixed span of time T you need T/\Delta t steps, and the little errors accumulate. Multiply an error of size \Delta t^2 by a number of steps proportional to 1/\Delta t and you get a global error of order \Delta t. Euler is a first-order method: halve the step size and you (roughly) halve the final error. That is a painfully slow bargain — to get one more decimal digit of accuracy you must take ten times as many steps.

Watch Euler in action — and watch it cheat

Let's integrate the friendliest system in physics: a mass on a spring, the simple harmonic oscillator. The force is F = -kx, so the acceleration is a = -\tfrac{k}{m}x — always pulling back toward the origin. Take k = m = 1, release the mass from x = 1 at rest, and the true motion is a perfect cosine that swings forever between -1 and +1. Its energy E = \tfrac12 m v^2 + \tfrac12 k x^2 should sit rock-steady at 0.5 for all time. Run Euler and read the last column carefully:

// Explicit (forward) Euler on a spring: a = -(k/m) x const k = 1, m = 1; const dt = 0.1; let x = 1, v = 0; // released from rest at x = 1 const energy = () => 0.5 * m * v * v + 0.5 * k * x * x; console.log("Euler — energy SHOULD stay at 0.500"); console.log(" t x v E"); for (let n = 0; n <= 60; n++) { if (n % 10 === 0) { console.log( `${(n * dt).toFixed(1).padStart(4)} ` + `${x.toFixed(3).padStart(7)} ` + `${v.toFixed(3).padStart(7)} ` + `${energy().toFixed(3).padStart(6)}`, ); } const a = -(k / m) * x; // acceleration from the CURRENT position const xNext = x + v * dt; // both updates use the OLD values const vNext = v + a * dt; x = xNext; v = vNext; }

The energy climbs, step after step — the oscillation slowly spirals outward instead of tracing a closed loop. Nothing is wrong with the code; this is Euler being Euler. Each straight-line step overshoots the true curved path just slightly outward, and for this system those overshoots always add energy rather than remove it. Left running for a few hundred steps the amplitude balloons absurdly. A method that invents energy from nothing is a serious problem if you were hoping to model, say, a stable planetary orbit.

Seeing the drift: energy over time

The plot below runs both Euler and the fancier RK4 method (coming up next) on that same unit spring and tracks the energy as time marches on. The true energy is the flat line at 0.5. Watch Euler peel away from it while RK4 clings to it.

Same equation, same step size — wildly different faithfulness. The gap is not about doing more steps; it's about being smarter within each step. That is exactly what Runge and Kutta figured out.

Runge–Kutta 4: sample the slope, then average it wisely

Euler's flaw is that it commits to the slope measured at the start of the step and rides it blindly all the way across, even though the true slope is changing the whole time. The fourth-order Runge–Kutta method (RK4), published around 1900, fixes this by taking four trial peeks at the slope inside a single step and blending them into one superior estimate. Writing the state's rate of change as f(\mathbf{y}, t), one RK4 step from \mathbf{y}_n is:

\begin{aligned} k_1 &= f\!\left(\mathbf{y}_n,\; t_n\right) &&\text{slope at the start} \\ k_2 &= f\!\left(\mathbf{y}_n + \tfrac{\Delta t}{2}k_1,\; t_n + \tfrac{\Delta t}{2}\right) &&\text{slope at the midpoint, using } k_1 \\ k_3 &= f\!\left(\mathbf{y}_n + \tfrac{\Delta t}{2}k_2,\; t_n + \tfrac{\Delta t}{2}\right) &&\text{midpoint again, using the better } k_2 \\ k_4 &= f\!\left(\mathbf{y}_n + \Delta t\,k_3,\; t_n + \Delta t\right) &&\text{slope at the end, using } k_3 \end{aligned}

Then step forward using a weighted average of the four, leaning hardest on the two midpoint estimates:

\mathbf{y}_{n+1} = \mathbf{y}_n + \frac{\Delta t}{6}\Bigl(k_1 + 2k_2 + 2k_3 + k_4\Bigr).

The weights \tfrac16, \tfrac13, \tfrac13, \tfrac16 are not plucked from the air: they are precisely the numbers that make the method's Taylor expansion agree with the true solution all the way out to the \Delta t^4 term. That buys an enormous prize. RK4's local error is O(\Delta t^5) and its global error is O(\Delta t^4). Halve the step and the error drops by a factor of 2^4 = 16, not a measly factor of 2. RK4 costs four force evaluations per step instead of Euler's one, but for any accuracy you actually care about it wins by a landslide — which is why it is the reliable workhorse behind countless simulators.

// RK4 on the SAME spring. State is the tuple y = [x, v]; f(y) = [v, -(k/m) x]. const k = 1, m = 1, dt = 0.1; type State = [number, number]; const deriv = ([x, v]: State): State => [v, -(k / m) * x]; const step = (y: State, s: State, h: number): State => [y[0] + h * s[0], y[1] + h * s[1]]; const energy = ([x, v]: State) => 0.5 * m * v * v + 0.5 * k * x * x; let y: State = [1, 0]; console.log("RK4 — energy stays glued to 0.500"); console.log(" t x v E"); for (let n = 0; n <= 60; n++) { if (n % 10 === 0) { console.log( `${(n * dt).toFixed(1).padStart(4)} ` + `${y[0].toFixed(3).padStart(7)} ` + `${y[1].toFixed(3).padStart(7)} ` + `${energy(y).toFixed(3).padStart(6)}`, ); } const k1 = deriv(y); const k2 = deriv(step(y, k1, dt / 2)); const k3 = deriv(step(y, k2, dt / 2)); const k4 = deriv(step(y, k3, dt)); y = [ y[0] + (dt / 6) * (k1[0] + 2 * k2[0] + 2 * k3[0] + k4[0]), y[1] + (dt / 6) * (k1[1] + 2 * k2[1] + 2 * k3[1] + k4[1]), ]; }

Read the energy column: it holds at 0.500 to three decimals across the whole run. Four cheap peeks per step, and the spurious drift essentially vanishes.

A real trajectory: a planet in orbit

Now for something with no closed-form shortcut in general — a body falling around a star under gravity. The acceleration points toward the centre and weakens with the square of the distance:

\mathbf{a} = -\,\frac{GM}{r^2}\,\hat{\mathbf{r}} = -\,\frac{GM}{r^3}\,\mathbf{r}, \qquad r = |\mathbf{r}|.

The state is now four numbers (x, y, v_x, v_y). We'll step it with symplectic Euler — a one-line twist on plain Euler that updates the velocity first and then advances the position using that new velocity. That tiny reordering makes the method conserve energy over the long haul (up to a small bounded wobble), which is exactly what you want for an orbit that should stay closed. Set GM = 1 and launch the body from radius 1 with just the right sideways speed for a circle; the energy should hover near E = \tfrac12 v^2 - \tfrac{GM}{r} = -0.5.

// A 2-D gravitational orbit, stepped with SYMPLECTIC Euler (velocity first, then position). const GM = 1, dt = 0.05; let x = 1, y = 0, vx = 0, vy = 1; // circular orbit: v = sqrt(GM/r) = 1 at r = 1 const energy = () => { const r = Math.sqrt(x * x + y * y); return 0.5 * (vx * vx + vy * vy) - GM / r; // should sit near -0.500 }; console.log("Symplectic-Euler orbit — bound, energy near -0.500, r near 1"); console.log(" t x y r E"); const steps = 126; // ~ one full revolution (period 2π) for (let n = 0; n <= steps; n++) { const r = Math.sqrt(x * x + y * y); if (n % 21 === 0) { console.log( `${(n * dt).toFixed(2).padStart(5)} ` + `${x.toFixed(3).padStart(7)} ` + `${y.toFixed(3).padStart(7)} ` + `${r.toFixed(3).padStart(7)} ` + `${energy().toFixed(3).padStart(7)}`, ); } const ax = -GM * x / (r * r * r); const ay = -GM * y / (r * r * r); vx += ax * dt; // 1) velocity uses the current acceleration... vy += ay * dt; x += vx * dt; // 2) ...position uses the freshly-updated velocity y += vy * dt; }

The radius stays near 1 and after roughly one period (t \approx 2\pi \approx 6.28) the body arrives back near its launch point: a genuine bound orbit, computed from nothing but the force law and a few hundred additions. Try plain forward Euler on this same setup and the planet spirals slowly out into deep space — a fake, and a catastrophic one for anything meant to run for millions of steps.

The trap: "smaller \Delta t always means a better answer." It feels obvious — tinier steps track the curve more faithfully, so shrink \Delta t as far as you can and the error must keep falling. It does, for a while. But every step also does floating-point arithmetic, and each operation carries a whisper of round-off error. Halve \Delta t and you double the number of steps, so you double the round-off you accumulate. Past a certain sweet-spot step size, the truncation error you're driving down is overtaken by round-off error you're piling up, and the total error starts getting worse as you shrink the step — all while the computation takes far longer. There is an optimum \Delta t, not a limit of "as small as possible."

A companion confusion worth clearing up: local error is not global error. Euler's per-step (local) error is O(\Delta t^2), which sounds respectable — but because the number of steps grows like 1/\Delta t, the errors accumulate into a global error of only O(\Delta t). Always quote the global order (Euler 1, RK4 4) when you compare methods; the per-step figure flatters everyone by one power.

Because accuracy over one step is not the same as good behaviour over a billion steps. RK4 is beautifully accurate locally, yet run for astronomically many steps it still slowly bleeds or gains energy, and a planetary system left to simmer for a simulated age will quietly drift off its true path. What long-run simulations really want is a method that conserves energy on average forever, even if any single step is a touch less accurate. Those are the symplectic integrators — they respect the deep geometric structure of Newtonian mechanics (they preserve area in the position–velocity phase space), so their energy error stays bounded and oscillating instead of marching off to infinity. The symplectic Euler we used for the orbit is the baby of the family; its famous, only-slightly-bigger sibling is the leapfrog (a.k.a. Verlet) integrator, the quiet default powering everything from galaxy formation codes to the molecular-dynamics engines that fold proteins. It updates position and velocity in an interleaved, time-symmetric "leapfrog" — and it is second-order accurate for the price of a single force evaluation per step. A story for its own page.