The Maximum Principle

With the Hamiltonian H = L + \lambda^\top f in hand, Pontryagin's maximum principle states the conditions an optimal trajectory must satisfy. It converts "minimise J over all admissible controls" — an infinite-dimensional search — into a set of differential equations and a pointwise minimisation of H. This page states those conditions and works the smallest possible example; the next page derives them.

Pontryagin's necessary conditions

Suppose (x^\*, u^\*) minimises J = \varphi(x(T)) + \int_0^T L\,dt subject to \dot{x} = f(x, u, t), x(0) = x_0, with the terminal state free. Then there is a costate \lambda(t) for which the optimal pair satisfies all of the following.

State equation — recovers the dynamics: \dot{x} = \frac{\partial H}{\partial \lambda} = f(x, u, t), \qquad x(0) = x_0.
Costate equation — runs backward in time: \dot{\lambda} = -\frac{\partial H}{\partial x}.
Minimum condition — at each instant the optimal control globally minimises the Hamiltonian over the admissible set \mathcal{U}: u^\*(t) = \arg\min_{u \in \mathcal{U}} H\big(x^\*, u, \lambda, t\big). When the minimiser is interior and H is smooth, this reduces to \partial H/\partial u = 0.
Transversality — fixes the costate at the free end: \lambda(T) = \frac{\partial \varphi}{\partial x}\big(x(T)\big).

The name is a quirk of sign convention. Pontryagin defined his Hamiltonian with the opposite sign (\mathcal{H} = -L + \lambda^\top f = -H), so minimising our H is the same as maximising his \mathcal{H} — hence "maximum principle". The condition is the same either way: pick the control that makes the Hamiltonian most favourable at every instant.

Necessary is not the same as sufficient. Just as ordinary calculus's "set the derivative to zero" only finds candidates for a minimum — a stationary point could be a maximum, a minimum, or a saddle — Pontryagin's four conditions only narrow the search. A control-costate pair satisfying all of them is a genuine candidate for optimality, often called an extremal, but the maximum principle alone does not certify that it truly minimises J.

Confirming optimality typically needs extra work: checking convexity of H in (x, u) (a Mangasarian-type sufficiency argument), comparing among several extremals, or a second-order test. Declaring victory the moment the four conditions hold — without that extra check — is one of the most common mistakes when applying the principle.

A two-point boundary value problem

Look at where the data lives. The state condition is given at the start: x(0) = x_0. The costate condition is given at the end: \lambda(T) = \partial\varphi/\partial x. The two ODEs are coupled — \dot{x} needs u^\*, which the minimum condition reads off from \lambda, while \dot{\lambda} needs x:

\begin{aligned} \dot{x} &= \tfrac{\partial H}{\partial \lambda}, & x(0) &= x_0, \\ \dot{\lambda} &= -\tfrac{\partial H}{\partial x}, & \lambda(T) &= \tfrac{\partial \varphi}{\partial x}. \end{aligned}

Because half the conditions sit at t = 0 and half at t = T, this is a two-point boundary value problem, not a plain initial-value problem you can integrate straight through. The state flows forward from x_0; the costate flows backward from \lambda(T); and they must be consistent in between. Solving the two together is the computational heart of optimal control.

The smallest worked example

Minimise the control effort J = \int_0^T u^2\,dt for the scalar system \dot{x} = u driven from x(0) = 0 to a fixed target x(T) = x_f. Here L = u^2, f = u, and there is no terminal cost.

Step 1 — write the Hamiltonian.

H = L + \lambda f = u^2 + \lambda u.

Step 2 — minimum condition. H is a smooth function of u, so set \partial H/\partial u = 0:

\frac{\partial H}{\partial u} = 2u + \lambda = 0 \quad\Longrightarrow\quad u = -\tfrac12 \lambda.

Step 3 — costate equation. H contains no bare x, so \partial H/\partial x = 0 and

\dot{\lambda} = -\frac{\partial H}{\partial x} = 0 \quad\Longrightarrow\quad \lambda = \text{const}.

Step 4 — therefore the control is constant. A constant \lambda makes u = -\tfrac12\lambda constant too. The state then grows at a constant rate:

\dot{x} = u = \text{const} \quad\Longrightarrow\quad x(t) = u\,t.

Step 5 — fit the boundary condition. Require x(T) = u\,T = x_f, so

u = \frac{x_f}{T}, \qquad x(t) = \frac{x_f}{T}\,t, \qquad \lambda = -2u = -\frac{2 x_f}{T}.

The minimum-effort path to the target is a straight line travelled at constant speed — exactly what intuition expects, and a reassuring first check of the machine. Here the endpoint is pinned, so \lambda(T) is set by the boundary condition x(T) = x_f rather than by transversality; the free-endpoint case would instead impose \lambda(T) = \partial\varphi/\partial x.

State and costate, side by side

Below are the optimal x(t) and \lambda(t) for the example. Slide the target x_f and the horizon T: the state rises in a straight line to its target, and the costate sits at the constant value -2 x_f / T. Reaching further or moving faster steepens the line and pushes the costate further from zero — a bigger shadow price for a harder transfer.

These four conditions are not assertions to memorise — each one falls out of making the augmented cost \bar{J} = \varphi + \int(H - \lambda^\top \dot{x})\,dt stationary, exactly as the Euler–Lagrange equation falls out of making a functional stationary. The state equation is stationarity in \lambda, the costate equation is stationarity in x (after an integration by parts), the minimum condition is stationarity in u, and the transversality condition is the leftover boundary term. The next page carries out that derivation in full.