Bang-Bang Control

The minimum condition asks the optimal control to minimise the Hamiltonian H at every instant. When H is a smooth bowl in u the minimum is interior and \partial H/\partial u = 0 pins it down. But a great many real problems are linear in a bounded control — and then the minimum lives at an edge, not an interior point. The result is bang-bang control: the input slams between its limits.

Minimising a straight line over an interval

Suppose the control enters the dynamics linearly and is box-constrained, |u| \le u_{\max}. Then the Hamiltonian is affine (a straight line) in u:

H(x, u, \lambda, t) = H_0(x, \lambda, t) + \sigma(t)\,u, \qquad \sigma(t) := \frac{\partial H}{\partial u}.

The coefficient of u is the switching function \sigma(t) — everything in H that multiplies the control. Minimising a straight line over the interval [-u_{\max}, u_{\max}] is trivial: a line has no interior minimum, so the smallest value sits at one endpoint, chosen by the sign of the slope:

if \sigma(t) > 0, then \sigma u is smallest at u = -u_{\max};
if \sigma(t) < 0, then \sigma u is smallest at u = +u_{\max}.

Both cases collapse into one rule:

u^\*(t) = -u_{\max}\,\operatorname{sign}\big(\sigma(t)\big).

The control is always pinned to a limit, never in between, and it switches — jumps from one limit to the other — precisely when \sigma(t) changes sign (crosses zero). That on-off, full-left / full-right behaviour is what "bang-bang" names.

If H is linear in a bounded control |u| \le u_{\max}, the optimal control takes only the extreme values \pm u_{\max}.
The choice is governed by the switching function \sigma = \partial H/\partial u via u^\* = -u_{\max}\,\operatorname{sign}(\sigma).
The control switches exactly where \sigma(t) crosses zero.

Worked example: the double integrator

Take a unit mass on a line pushed by a bounded force, \ddot{x} = u with |u| \le 1. Write it as a first-order system with state x_1 = position, x_2 = velocity:

\dot{x}_1 = x_2, \qquad \dot{x}_2 = u.

Step 1 — Hamiltonian. For a generic running cost the Hamiltonian is H = L + \lambda_1 x_2 + \lambda_2 u. The only term carrying u is \lambda_2 u, so the switching function is

\sigma(t) = \frac{\partial H}{\partial u} = \lambda_2(t).

Step 2 — costate equations. Differentiate -H in the state components:

\dot{\lambda}_1 = -\frac{\partial H}{\partial x_1} = 0, \qquad \dot{\lambda}_2 = -\frac{\partial H}{\partial x_2} = -\lambda_1.

Step 3 — solve them. The first says \lambda_1 is constant; feeding that into the second gives a costate that is linear in time:

\lambda_1 = c_1, \qquad \lambda_2(t) = c_2 - c_1 t \;=\; \sigma(t).

Step 4 — count the switches. The switching function \sigma(t) = c_2 - c_1 t is a straight line in t, so it changes sign at most once on the interval. Hence the time-optimal control is

u^\*(t) = -\operatorname{sign}\big(\sigma(t)\big) = \pm 1, \qquad \text{with at most one switch.}

Full thrust one way, a single flip, full thrust the other way — that is the entire optimal strategy for the double integrator. The next page turns this into the famous switching curve in the phase plane.

Watch the control switch

Below, the switching function is the straight line \sigma(t) = \sigma_0 + s\,t (slide its intercept \sigma_0 and slope s), and the control is u(t) = -\operatorname{sign}(\sigma(t)) with u_{\max} = 1. Where \sigma crosses zero the control jumps instantly between +1 and -1 — there is no gentle ramp, only the bang. With a sign change the control switches once; without one (the line stays one side of zero) it never switches and the control is constant.

The bang-bang rule needs \sigma(t) to have a definite sign. What if \sigma \equiv 0 over a whole interval, not just at an instant? Then the minimum condition says nothing about u there — the Hamiltonian is flat in u, indifferent to its value. Such a stretch is a singular arc, and the control on it is found by repeatedly differentiating \sigma = 0 in time until u reappears, then solving for it. Singular arcs are the subtle exception to "always at a limit"; for the double integrator \sigma(t) is a non-constant line, so it is zero only at an instant and no singular arc occurs.