Data Hazards and Forwarding

The clean five-stage pipeline told a comforting lie: that instructions are independent, so we can overlap them freely. Real programs are not so polite. Consider two lines any compiler emits a thousand times:

add r1, r2, r3 # I1: r1 = r2 + r3 sub r4, r1, r5 # I2: r4 = r1 - r5 (needs r1!)

I2 needs the r1 that I1 is still computing. This is a data hazard: an instruction depends on a result that an earlier, still-in-flight instruction has not yet made available. Left alone, the pipeline would hand I2 a stale r1 and silently compute the wrong answer. This lesson is about how the hardware notices, and the beautiful trick that fixes it almost for free.

The timing mismatch — read before write

Trace the cycles. I1 only produces r1 at the end of its EX stage (cycle 3), and only writes it back to the register file in WB (cycle 5). But I2, following one cycle behind, wants r1 as an input to its own EX in cycle 4 — a full cycle before I1 writes it. This is the classic read-after-write (RAW) hazard, and the gap is the enemy:

\underbrace{\text{I1 result ready (end EX)}}_{\text{cycle 3}} \;\longrightarrow\; \underbrace{\text{I2 needs it (start EX)}}_{\text{cycle 4}} \;\ll\; \underbrace{\text{I1 writes register}}_{\text{cycle 5 (WB)}}.

Read that chain carefully: the value exists in cycle 3, sitting in the EX/MEM pipeline register. It is just not in the register file until cycle 5. The naive fix — stall I2 until I1's WB finishes — would cost two or three wasted cycles on nearly every instruction pair. Unacceptable. There is a better way.

Forwarding: don't wait, wire it

The value I2 wants already exists in cycle 3 — why route it the long way through the register file? Forwarding (also called bypassing) adds short-circuit wires and multiplexers that grab a freshly computed result straight out of the EX/MEM or MEM/WB pipeline register and feed it back into the ALU inputs of a following instruction. The result is delivered a cycle after it is born, exactly when the consumer's EX needs it. Step through the figure: first the hazard, then the bypass wire that erases it.

With a full forwarding network — paths from EX/MEM and MEM/WB back into EX — a back-to-back dependent pair of ALU instructions runs with zero stall cycles. The pipeline keeps its \text{CPI} = 1. Forwarding is why the simple pipeline is fast at all.

The one hazard forwarding can't beat: load-use

Forwarding works because an ALU result is ready at the end of EX. But a load's data isn't ready until the end of MEM — one stage later. So if the very next instruction needs that loaded value, even forwarding arrives too late:

lw r1, 0(r2) # r1 loaded from memory — ready only at end of MEM (cycle 4) add r4, r1, r5 # needs r1 at start of its EX (cycle 4) — impossible!

The data would have to travel backwards in time. It can't, so the hardware inserts exactly one bubble (a stall of one cycle), delaying the add so its EX lands in cycle 5 — and then forwarding from MEM/WB delivers r1 just in time. This unavoidable load-use stall is the reason compilers try to slot an independent instruction between a load and its first use.

a RAW hazard is a read-after-write dependence: the consumer needs a value the producer hasn't written back yet;
forwarding (EX/MEM → EX and MEM/WB → EX) resolves every ALU-to-ALU RAW hazard with zero stalls;
the load-use hazard is the one exception — it always costs exactly one bubble, because load data isn't ready until after MEM;
WAR and WAW hazards cannot occur here: instructions read in ID and write in WB, both strictly in program order.

Why WAR and WAW simply don't happen (yet)

A write-after-read (WAR) hazard means a later instruction overwrites a register before an earlier one has read it; a write-after-write (WAW) means two writes to the same register land out of order. In our in-order pipeline both are impossible, and the reason is structural: every instruction reads its operands in ID (early) and writes its result in WB (late), and instructions enter and leave in program order. An earlier instruction always reads before a later one writes, and always writes before a later one writes. These "name" hazards only appear once we let instructions execute out of order — a later module, where register renaming exists precisely to kill them.

A hazard detector, in code

The forwarding unit is really just a comparator: does this instruction read a register that a recent instruction is about to write? Here is the logic in miniature — it classifies each adjacent pair as independent, a forwardable ALU hazard, or a load-use stall.

type Insn = { op: "alu" | "load"; dst: number; srcs: number[] }; const prog: Insn[] = [ { op: "load", dst: 1, srcs: [2] }, // lw r1, 0(r2) { op: "alu", dst: 4, srcs: [1, 5] }, // add r4, r1, r5 <- uses r1 right after load { op: "alu", dst: 6, srcs: [4, 7] }, // add r6, r4, r7 <- uses r4 from an ALU op { op: "alu", dst: 8, srcs: [9, 10] }, // add r8, r9, r10 <- independent ]; let stalls = 0; for (let i = 1; i < prog.length; i++) { const prev = prog[i - 1], cur = prog[i]; const raw = cur.srcs.includes(prev.dst); if (raw && prev.op === "load") { stalls += 1; console.log(`I${i} -> I${i + 1}: load-use RAW on r${prev.dst} => 1 stall, then forward`); } else if (raw) { console.log(`I${i} -> I${i + 1}: ALU RAW on r${prev.dst} => forward, 0 stalls`); } else { console.log(`I${i} -> I${i + 1}: independent, no hazard`); } } console.log(`Total stall cycles inserted: ${stalls}`);

More than you'd guess. A result must be forwardable to both ALU inputs, from both the EX/MEM and MEM/WB registers, so each ALU operand gets a multiplexer choosing among the register-file value and several bypass sources. On a superscalar machine that issues four instructions per cycle, every one of the eight ALU inputs can come from any of a dozen recently produced results — the forwarding network becomes a dense crossbar that can dominate the wiring and even limit the clock. This is a recurring theme in architecture: the logic is cheap, but the bypass wiring that keeps everything fed is expensive. Wide out-of-order cores spend a startling fraction of their area just shuffling results around.

A tempting summary is "forwarding makes all data hazards free." Not quite. Forwarding kills the ALU-to-ALU RAW hazard because an ALU result exists at the end of EX. But a load produces its value one stage later, at the end of MEM, and no wire can send data back in time to the consumer's EX in the same cycle. So the load-use case always costs one bubble — unless the compiler hides it by scheduling an unrelated instruction into the gap. Whenever someone claims "forwarding removes every stall," ask them about the instruction right after a load.