The Phases of Compilation

The structure of a compiler told us where work lives — the front-end / back-end split by dependency. Now we follow the work through time. A compiler transforms a program by passing it down a pipeline of phases, each taking one representation and producing the next, more machine-like one. The classic seven, straight out of the Dragon Book, are:

\text{lexing} \to \text{parsing} \to \text{semantic analysis} \to \text{IR generation} \to \text{machine-independent optimisation} \to \text{code generation} \to \text{machine-dependent optimisation}.

Two more components stand beside the pipeline rather than inside it, because every phase talks to them. The symbol table records what each identifier means — its type, its scope, its storage — and is written by the early phases and read by all the later ones. The error handler collects diagnostics from every phase so the compiler can report as many problems as possible in one run rather than dying on the first mistake. Draw the seven phases as a vertical column and these two as tall bars spanning the whole column.

The pipeline at a glance

Before we trace a real statement, here is the whole journey as a table: what each phase consumes and what it produces. Notice how the representation grows steadily more machine-like as you go down — from characters, to a tree of meaning, to instructions.

PhaseInputOutputRegion
1. Lexical analysis (lexing)character streamtoken streamfront end
2. Syntax analysis (parsing)token streamparse tree / ASTfront end
3. Semantic analysisASTannotated AST (types, coercions)front end
4. Intermediate code generationannotated ASTIR (e.g. three-address code)front end
5. Machine-independent optimisationIRbetter IRmiddle end
6. Code generationIRtarget assembly / machine codeback end
7. Machine-dependent optimisationtarget codebetter target codeback end

Alongside all seven run the symbol table manager and the error handler, touched by every row.

One statement, all the way down

The best way to feel the pipeline is to push a single line through it. We use the Dragon Book's own running example — an assignment where position, initial and rate are floating-point variables:

position = initial + rate * 60

Watch each phase rewrite it.

Phase 1 — Lexing: characters become tokens

The lexer scans the raw characters and groups them into tokens, discarding whitespace. Each identifier becomes an id token carrying a pointer into the symbol table; each operator and literal becomes its own token.

\langle \mathbf{id},1\rangle\ \langle = \rangle\ \langle \mathbf{id},2\rangle\ \langle + \rangle\ \langle \mathbf{id},3\rangle\ \langle * \rangle\ \langle \mathbf{60} \rangle

Here \langle\mathbf{id},1\rangle means "an identifier token whose symbol-table entry is #1" — entry #1 being position, #2 initial, #3 rate.

Phase 2 — Parsing: tokens become a tree

The parser checks the tokens against the language grammar and builds a syntax tree that captures structure — crucially, that * binds tighter than +, so rate * 60 is a subtree of the +, which is in turn the right child of =.

= / \ <id,1> + / \ <id,2> * / \ <id,3> 60

Phase 3 — Semantic analysis: types and a coercion appear

Now the compiler checks meaning against the symbol table. Every variable is a float, but the literal 60 is an integer. Adding an integer to a float requires a coercion, so the analyser inserts an explicit inttofloat node — a change to the tree that no purely syntactic phase could make.

= / \ <id,1> + / \ <id,2> * / \ <id,3> inttofloat | 60

Phase 4 — IR generation: a linear three-address form

The annotated tree is flattened into three-address code — instructions with at most one operator and (roughly) three operands, using temporaries t_1, t_2, t_3:

t1 = inttofloat(60) t2 = id3 * t1 t3 = id2 + t2 id1 = t3

Phase 5 — Machine-independent optimisation

The optimiser notices that inttofloat(60) is a compile-time constant — fold it to 60.0 — and that t3 is used only to be copied into id1, so the copy can be eliminated. The IR shrinks:

t1 = id3 * 60.0 id1 = id2 + t1

Phase 6 — Code generation (with allocation)

The back end selects target instructions and assigns registers (R1, R2). The F suffix marks floating-point ops; #60.0 is an immediate:

LDF R2, id3 ; load rate MULF R2, R2, #60.0 ; R2 = rate * 60.0 LDF R1, id2 ; load initial ADDF R1, R1, R2 ; R1 = initial + rate*60.0 STF id1, R1 ; store into position

A final machine-dependent optimisation pass (phase 7) might reorder these to hide load latency or fuse the multiply-add into a single MADDF if the target has one — decisions that depend entirely on the specific chip. And that is the whole pipeline: one readable line of source became five machine instructions, transformed step by careful step.

The token stream, for real

Phase 1 is concrete enough to run. Below is a tiny hand-written lexer for exactly this class of statement. It walks the characters, emits a token for each identifier, number and operator, and prints the stream — the honest input to the parser.

type Tok = { kind: string; text: string }; function lex(src: string): Tok[] { const toks: Tok[] = []; let i = 0; const isAlpha = (c: string) => /[A-Za-z_]/.test(c); const isDigit = (c: string) => /[0-9]/.test(c); while (i < src.length) { const c = src[i]; if (c === " " || c === "\t") { i++; continue; } // skip whitespace if (isAlpha(c)) { // identifier let j = i + 1; while (j < src.length && (isAlpha(src[j]) || isDigit(src[j]))) j++; toks.push({ kind: "id", text: src.slice(i, j) }); i = j; } else if (isDigit(c)) { // number literal let j = i + 1; while (j < src.length && isDigit(src[j])) j++; toks.push({ kind: "num", text: src.slice(i, j) }); i = j; } else if ("=+*-/()".includes(c)) { // single-char operator toks.push({ kind: "op", text: c }); i++; } else { toks.push({ kind: "error", text: c }); // hand it to the error handler i++; } } return toks; } const stream = lex("position = initial + rate * 60"); for (const t of stream) console.log(`<${t.kind}, "${t.text}">`); console.log(`--- ${stream.length} tokens ---`);

No — and that is exactly why the Dragon Book draws them beside the pipeline as tall bars, not as boxes in the chain. A phase is a transformation that takes one representation to the next. The symbol table and the error handler are shared services that every phase consults. The lexer creates a symbol-table entry the first time it sees rate; the semantic analyser fills in its type; the code generator reads its storage location. If the error handler were a phase, a single lexical typo would stop the compiler before the parser ever ran — but good compilers recover and press on, precisely so they can report a whole batch of errors at once.

The two words are constantly muddled. A phase is a logical unit of work defined by what it computes — lexing, parsing, optimisation. A pass is a physical traversal: one complete read of the program (or its IR) from top to bottom. The mapping between them is many-to-many and is an engineering choice, not a law of nature.

A single-pass compiler (early Pascal is the classic) folds lexing, parsing, semantic analysis and code generation into one sweep — many phases, one pass — which is fast and memory-thin but forbids optimisations that need to see the whole program. A multi-pass compiler runs many passes: the optimiser alone may sweep the IR dozens of times, so one phase becomes many passes. So "how many phases?" and "how many passes?" have different answers, and neither determines the other. Count transformations for phases; count sweeps for passes.