Compiling with Continuations
On the last two pages we made data explicit — a function became a
closure
over an environment instead of borrowing the host's functions. The last of Reynolds' three
leaks remains: control. A direct-style interpreter still borrows the host's call stack
and its "return" mechanism — it never says, in its own terms, what happens next. To compile a
language faithfully, especially one with exceptions, generators, or proper tail calls, we must make
"what happens next" a value we can name, pass, and store. That value is a continuation,
and rewriting a program so every continuation is explicit is continuation-passing style
(CPS).
CPS is not merely a curiosity — it is one of the great intermediate representations of
compiler back ends. Guy Steele's 1978 RABBIT compiler for Scheme showed that if you transform
the whole program into CPS, then "lambda is the ultimate GOTO": every jump, every return,
every loop is a tail call to a continuation, and code generation becomes almost mechanical because
control flow is already spelled out. Modern compilers — SML/NJ, and via CPS's cousin ANF, GHC and many
others — are built on exactly this move. This page shows what a continuation is, how CPS makes control
explicit, why "administrative redexes" are its main nuisance, and how ANF is the tidier sibling most
real compilers prefer.
A continuation is "the rest of the computation"
Fix a spot inside an evaluating expression. Everything that will still happen after the value
at that spot is produced — that whole pending remainder, as a function of the value — is its
continuation. In 1 + (2 \times 3), once
2 \times 3 yields 6, the rest is "add 1 to it":
the continuation of the subexpression 2\times 3 is
\kappa = \lambda v.\;1 + v. CPS turns this implicit "rest" into an explicit
extra argument that every function and primitive receives and calls instead of returning:
\mathcal{C}[\![ e_1 + e_2 ]\!]\,\kappa \;=\; \mathcal{C}[\![ e_1 ]\!]\,\Big(\lambda a.\;\mathcal{C}[\![ e_2 ]\!]\,\big(\lambda b.\;\kappa\,(a + b)\big)\Big)
Read it aloud: to compute e_1 + e_2 and then do
\kappa with the sum, first compute e_1 and hand
its value a to a continuation that computes e_2,
hands its value b to a continuation that finally calls
\kappa on a+b. Two consequences fall out at once,
and they are why compilers love CPS: evaluation order is now written down (left before
right, because that is the nesting of the continuations), and every call is in tail
position — nothing ever waits on the stack for a call to return, because there are no returns.
CPS linearises control into a chain of tail calls
The direct-style expression (a + b) \times c hides its control flow in the
tree structure — "do the inner add, then the multiply" is implicit. CPS makes it a straight
chain: compute the sum, pass it forward; compute the product, pass it forward; hand the
result to the halt continuation. Each arrow is a tail call — a jump, not a
call-that-returns. Reveal the chain:
That straight chain is the point. A code generator can walk it and emit one basic block after another,
each ending in a jump to the next — which is precisely what machine code looks like. Control flow that
was structural (nested subtrees, implicit returns) is now syntactic (a sequence of
named steps), and that is what "compiling with continuations" buys you.
The CPS converter, running
Here is a real (if small) CPS transformer for an arithmetic-and-let language, plus a CPS
evaluator that threads an explicit continuation and never returns a value the ordinary way —
it always calls its continuation. The transformer emits the nested-lambda CPS text so you can see the
structure; the evaluator proves it computes the right answer. Press Run:
type Expr =
| { kind: "lit"; n: number }
| { kind: "var"; name: string }
| { kind: "add"; left: Expr; right: Expr }
| { kind: "mul"; left: Expr; right: Expr };
// --- 1) CPS TRANSFORM to text: C[e] k. k is a string naming the continuation. ---
let counter = 0;
const fresh = () => "v" + counter++;
function cps(e: Expr, k: (val: string) => string): string {
switch (e.kind) {
case "lit": return k(String(e.n));
case "var": return k(e.name);
case "add":
case "mul": {
const op = e.kind === "add" ? "+" : "*";
return cps(e.left, (a) =>
cps(e.right, (b) => {
const r = fresh();
// "let r = a op b in " — the administrative structure of CPS.
return `let ${r} = ${a} ${op} ${b} in ${k(r)}`;
}));
}
}
}
// --- 2) CPS EVALUATOR: no returns; every step calls its continuation k. ---
function evalk(e: Expr, env: Record<string, number>, k: (n: number) => void): void {
switch (e.kind) {
case "lit": k(e.n); return;
case "var": k(env[e.name]); return;
case "add": evalk(e.left, env, (a) => evalk(e.right, env, (b) => k(a + b))); return;
case "mul": evalk(e.left, env, (a) => evalk(e.right, env, (b) => k(a * b))); return;
}
}
// (a + b) * c with a=2, b=3, c=4
const expr: Expr = { kind: "mul",
left: { kind: "add", left: { kind: "var", name: "a" }, right: { kind: "var", name: "b" } },
right: { kind: "var", name: "c" } };
console.log("CPS form: " + cps(expr, (v) => v)); // the named, sequentialised program
evalk(expr, { a: 2, b: 3, c: 4 }, (result) =>
console.log("halt continuation receives:", result)); // 20
The printed CPS form — let v0 = a + b in let v1 = v0 * c in v1 — has named every
intermediate and fixed the order. The evaluator never writes return someValue to its
caller; it calls k. The final answer arrives at the halt continuation. Control has become
data.
Administrative redexes, and why ANF is the practical cousin
The naïve CPS transform sprinkles in extra \lambdas and immediately applies
them — (\lambda a.\,\ldots)\,e_1. These administrative redexes
are bookkeeping artefacts of the transform, not part of the source program's meaning, and if you leave
them in, the CPS code is bloated and slow. Plotkin (1975) identified them; the fix is either a
one-pass "higher-order" CPS transform that reduces them as it goes (the trick in Danvy &
Filinski's compact converters), or to skip CPS's redundancy altogether and target
A-Normal Form (ANF).
-
ANF names every intermediate result with a
let and requires
arguments to be atomic (a variable or constant), so a program becomes a flat sequence of
let x = <primitive op> in … — exactly the let v0 = a+b in let v1 = v0*c
in v1 above. It keeps the direct-style "returns" but forbids nested non-trivial subexpressions.
-
CPS goes further: it also reifies the return points as continuation
functions, so there are no returns at all — only tail calls. It exposes control (call/cc,
exceptions, coroutines) more directly, at the cost of those administrative redexes.
-
They are close relatives. ANF is essentially "CPS without explicitly naming the
continuations" — Sabry & Felleisen showed the two are inter-translatable. Many compilers (GHC's
Core, and Flanagan et al.'s classic result "the essence of compiling with continuations") use ANF
because it gives most of CPS's benefits with less clutter.
Whichever you pick, the deliverable to the code generator is the same: a program where every operation
is named, every argument is trivial, and control flows as an explicit sequence of steps — a shape that
maps almost one-to-one onto basic blocks and jumps.
Why tail calls fall out for free
In CPS there are no non-tail calls — every call is the last thing its function does, because
the "what to do with the result" was passed along as \kappa rather than
waited for. So a function call compiles to a jump that reuses the current stack frame,
and a loop written as tail recursion runs in constant space with no special "tail-call optimisation"
pass — the CPS form is the optimisation.
\mathcal{C}[\![ f\;e ]\!]\,\kappa \;=\; \mathcal{C}[\![ e ]\!]\,\big(\lambda v.\; f\;v\;\kappa\big)
Look at the call f\;v\;\kappa: the function receives the argument
and the continuation, and it is in tail position in the continuation lambda. There is nothing
after it to come back to. This is the technical content of Steele's slogan "lambda is the ultimate
GOTO": once control is explicit, a procedure call and a goto are the same instruction, and the compiler
stops treating them differently. That single uniformity is why a CPS-based back end can generate tight,
loop-friendly code without a separate control-flow analysis.
In 1976–78 Guy Steele and Gerald Sussman wrote a series of MIT AI-Lab memos — the "Lambda Papers" —
with irresistible titles: Lambda: The Ultimate Imperative, …the Ultimate Declarative,
…the Ultimate GOTO. Steele's master's thesis turned the ideas into RABBIT, the
first compiler to use CPS as its intermediate language, demonstrating that Scheme's function calls could
compile to machine jumps as efficient as any hand-written loop. The lineage continued with
ORBIT (Kranz et al., for the T dialect) and Appel's book Compiling with
Continuations (1992), which made CPS the backbone of the Standard ML of New Jersey compiler. The
provocative claim underneath all of it — that a function call is just a goto that passes arguments — is
now simply how compilers think.
-
The naïve CPS transform produces administrative redexes. Those extra
apply-a-lambda-immediately terms are transform bookkeeping, not source meaning; left unreduced they
bloat the code and cost speed. Use a one-pass/higher-order CPS converter that reduces them on the fly,
or target ANF, which never introduces them.
-
CPS makes evaluation order explicit — so you must get it right in the transform.
Which continuation you nest inside which is the language's evaluation order. Nesting the
right operand's continuation inside the left's gives left-to-right; swap them and you have silently
defined a different, right-to-left language.
-
ANF is not "CPS with different syntax" — it keeps returns. ANF names intermediates
and demands atomic arguments but still uses direct-style returns; CPS abolishes returns entirely in
favour of tail calls to continuations. They are inter-translatable, but don't conflate them: pick ANF
for a leaner IR, CPS when you must reify control (call/cc, first-class continuations, coroutines).