Hazards and Forwarding in RTL

Last lesson ended on a cliffhanger: our beautiful five-stage pipeline computed 0 where the answer was 12. Nothing was miswired. The machine did exactly what we built: add x3, x1, x2 read its sources in ID at cycle 3, while the addi writing x1 was still two stages from write-back. The value existed — it was sitting right there in a pipeline register — but nobody had built a road to it. This is a RAW (read-after-write) data hazard, and this lesson builds the roads: a forwarding unit of comparators and muxes, a one-cycle stall for the one case forwarding cannot save, and a flush for branches. All of it is hardware — the hazard theory you know, now as gates.

The broken trace, dissected

Watch the dependent pair move through the pipeline. addi x1, x0, 5 (call it I0) reaches WB at cycle 4. But add x3, x1, x2 (I2) read the register file back in cycle 3 — it got the stale x1 = 0, and by cycle 4 it is in EX faithfully adding garbage. The fix is hiding in plain sight: at the moment I2's ALU fires (cycle 4), I0's freshly computed 5 is sitting in the MEM/WB pipeline register, and I1's 7 is sitting in EX/MEM. The correct values are on the chip, centimetres of wire away — just not in the register file yet. So: don't wait for the register file. Wire the pipeline registers straight back to the ALU inputs. That road is called a bypass, or forwarding path.

The picture: roads back through time

Here is the standard diagram every architect doodles — instructions as rows, cycles as columns, and the forwarding paths as arrows running backwards against the flow:

Note what the arrows land on: always the EX stage of the consumer. Forwarding serves the ALU; that is where values are consumed, so that is where the muxes live.

The forwarding unit: comparators and muxes

As hardware, forwarding is almost humble: for each ALU operand, compare the consumer's source register against the destination of the instructions ahead, and steer a mux. In RTL:

// The forwarding unit: comparators steering muxes. Note the THREE-part // qualification: the producer must write, its rd must match, and rd != x0. always_comb begin fwd_a = 2'b00; // default: the RF value if (ex_mem.ctl.reg_write && ex_mem.rd != 5'd0 && ex_mem.rd == id_ex.rs1) fwd_a = 2'b10; // newest wins: EX/MEM first else if (mem_wb.ctl.reg_write && mem_wb.rd != 5'd0 && mem_wb.rd == id_ex.rs1) fwd_a = 2'b01; end // (fwd_b: same, with rs2) // The bypass mux sits right in front of the ALU's A input: assign alu_a = (fwd_a == 2'b10) ? ex_mem.alu_out // one cycle ahead of the RF : (fwd_a == 2'b01) ? mem_wb.wb_data // two cycles ahead : id_ex.a; // no hazard: the RF was right

Two subtleties are doing heavy lifting. Priority: if both older instructions match (add x1,… then add x1,… then a reader of x1), EX/MEM must win — it is the younger of the two producers, holding the newest value. And qualification: a match on register number alone is not enough — more on that in the warning below, where a single missing term becomes this module's favourite bug.

The comparator ex_mem.rd == id_ex.rs1 is a liar unless you qualify it twice. Qualify with reg_write: SW and BEQ write no register, and for them the bits at position 11:7 are immediate bits — an unqualified match happily forwards a store's address offset into someone's ALU. Qualify with rd != x0: jal x0, loop and friends "write" x0; an instruction that legitimately reads x0 (expecting a rock-solid 0) would be handed the jump's link address instead. Three terms, always: writes ∧ rd ≠ 0 ∧ rd = rs. Drop one and the CPU passes every simple test — dependent arithmetic works! — then corrupts a register the first time a program jumps with x0. The final lesson injects exactly this bug on purpose, and watches the verification machinery catch it.

The stall: when the value does not exist yet

Forwarding moves values through space; it cannot move them backwards in time. LW's data leaves the data memory at the end of its MEM stage — but a consumer in the very next slot needs it in EX during that same cycle. No wire fixes that. The only cure is to make the consumer wait one cycle:

// Load-use detection: the instruction in EX is a load whose rd matches a // source of the instruction still in ID. Nothing can forward what does not // exist yet -- so stall one cycle. assign load_use = id_ex.ctl.mem_read && id_ex.rd != 5'd0 && (id_ex.rd == if_id_rs1 || id_ex.rd == if_id_rs2); // Stalling = freeze the front, bubble the middle: // pc <= pc; if_id <= if_id; (hold: replay the same fetch) // id_ex <= '0; (inject NOP controls into EX) // Taken branch, resolved in EX: squash the two younger instructions. assign flush = ex_branch_taken; // pc <= branch_target; // if_id <= '0; id_ex <= '0; (both become bubbles)

A stall is three coordinated acts, all in always_ff enables: the pc and IF/ID register hold (the stalled instruction replays its decode), and the ID/EX register receives an all-zero control bundle — a bubble, an instruction-shaped nothing that flows harmlessly down the pipe. One cycle later the load's data is in MEM/WB and ordinary forwarding finishes the job. Compilers help by scheduling an unrelated instruction into the load's shadow, but the hardware interlock must exist regardless.

Branches are the same medicine at a different address. Our BEQ resolves in EX — by which time the two instructions behind it (predicted not taken, fetched sequentially) are already in IF and ID. If the branch is taken, both are flushed: converted to bubbles, their side effects never having happened, while the pc redirects to the target. A taken branch thus costs two dead cycles — the seed of the entire branch-prediction industry.

The complete pipeline, healed

Everything at once now: the simulator gains a forwarding unit (with its three-term qualification), load-use detection, and branch flush — then runs the program that broke last lesson, extended with a store, a genuine load-use pair, and a taken branch with two doomed instructions in its shadow. Every forward, stall and flush is logged as it happens:

const FORWARDING = true; // flip to false to relive lesson 38's bug // -- Assembler + decoder + control (compact recap). -- function R(f7: number, f3: number, rd: number, rs1: number, rs2: number): number { return ((f7 << 25) | (rs2 << 20) | (rs1 << 15) | (f3 << 12) | (rd << 7) | 0x33) | 0; } function I(op: number, f3: number, rd: number, rs1: number, imm: number): number { return (((imm & 0xfff) << 20) | (rs1 << 15) | (f3 << 12) | (rd << 7) | op) | 0; } function S(f3: number, rs2: number, rs1: number, imm: number): number { return ((((imm >> 5) & 0x7f) << 25) | (rs2 << 20) | (rs1 << 15) | (f3 << 12) | ((imm & 0x1f) << 7) | 0x23) | 0; } function B(f3: number, rs1: number, rs2: number, imm: number): number { return ((((imm >> 12) & 1) << 31) | (((imm >> 5) & 0x3f) << 25) | (rs2 << 20) | (rs1 << 15) | (f3 << 12) | (((imm >> 1) & 0xf) << 8) | (((imm >> 11) & 1) << 7) | 0x63) | 0; } const asm = { ADD: (rd: number, rs1: number, rs2: number) => R(0x00, 0, rd, rs1, rs2), ADDI: (rd: number, rs1: number, imm: number) => I(0x13, 0, rd, rs1, imm), LW: (rd: number, rs1: number, imm: number) => I(0x03, 2, rd, rs1, imm), SW: (rs2: number, rs1: number, imm: number) => S(2, rs2, rs1, imm), BEQ: (rs1: number, rs2: number, imm: number) => B(0, rs1, rs2, imm), }; interface Decoded { name: string; rd: number; rs1: number; rs2: number; imm: number; useRs1: boolean; useRs2: boolean } function decode(w: number): Decoded { const opcode = w & 0x7f, rd = (w >> 7) & 0x1f, f3 = (w >> 12) & 7; const rs1 = (w >> 15) & 0x1f, rs2 = (w >> 20) & 0x1f, f7 = (w >>> 25) & 0x7f; const immI = w >> 20; const immS = ((w >> 25) << 5) | ((w >> 7) & 0x1f); const immB = ((w >> 31) << 12) | (((w >> 7) & 1) << 11) | (((w >> 25) & 0x3f) << 5) | (((w >> 8) & 0xf) << 1); if (opcode === 0x33) return { name: f3 === 0 ? (f7 === 0x20 ? "SUB" : "ADD") : f3 === 7 ? "AND" : "OR", rd, rs1, rs2, imm: 0, useRs1: true, useRs2: true }; if (opcode === 0x13) return { name: "ADDI", rd, rs1, rs2, imm: immI, useRs1: true, useRs2: false }; if (opcode === 0x03) return { name: "LW", rd, rs1, rs2, imm: immI, useRs1: true, useRs2: false }; if (opcode === 0x23) return { name: "SW", rd: 0, rs1, rs2, imm: immS, useRs1: true, useRs2: true }; return { name: "BEQ", rd: 0, rs1, rs2, imm: immB, useRs1: true, useRs2: true }; } interface Ctl { RegWrite: number; ALUSrc: number; ALUOp: string; MemRead: number; MemWrite: number; MemToReg: number; Branch: number } function control(d: Decoded): Ctl { const n = d.name; return { RegWrite: n === "SW" || n === "BEQ" ? 0 : 1, ALUSrc: n === "ADDI" || n === "LW" || n === "SW" ? 1 : 0, ALUOp: n === "SUB" || n === "BEQ" ? "SUB" : n === "AND" ? "AND" : n === "OR" ? "OR" : "ADD", MemRead: n === "LW" ? 1 : 0, MemWrite: n === "SW" ? 1 : 0, MemToReg: n === "LW" ? 1 : 0, Branch: n === "BEQ" ? 1 : 0, }; } interface IFID { pc: number; word: number } interface IDEX { pc: number; name: string; rd: number; rs1: number; rs2: number; useRs1: boolean; useRs2: boolean; a: number; b: number; imm: number; c: Ctl } interface EXMEM { pc: number; name: string; rd: number; alu: number; storeVal: number; c: Ctl } interface MEMWB { pc: number; name: string; rd: number; val: number; c: Ctl } interface State { pc: number; regs: number[]; mem: number[]; if_id: IFID | null; id_ex: IDEX | null; ex_mem: EXMEM | null; mem_wb: MEMWB | null; } function next(s: State, imem: number[], ev: string[]): State { const regs = s.regs.slice(), mem = s.mem.slice(); // -- WB (first half): write, so ID reads fresh values. -- if (s.mem_wb && s.mem_wb.c.RegWrite && s.mem_wb.rd !== 0) regs[s.mem_wb.rd] = s.mem_wb.val; // -- MEM. -- let mem_wb: MEMWB | null = null; if (s.ex_mem) { const m = s.ex_mem; let val = m.alu; if (m.c.MemWrite) mem[m.alu >> 2] = m.storeVal; if (m.c.MemRead) val = mem[m.alu >> 2] | 0; mem_wb = { pc: m.pc, name: m.name, rd: m.rd, val, c: m.c }; } // -- EX, with the forwarding unit in front of the ALU. -- let ex_mem: EXMEM | null = null; let taken = false, target = 0; if (s.id_ex) { const e = s.id_ex; // The three-term comparison: rd matches, rd is not x0, and the instruction really writes. const fwd = (rs: number, used: boolean, which: string): number | null => { if (!FORWARDING || !used || rs === 0) return null; if (s.ex_mem && s.ex_mem.c.RegWrite && s.ex_mem.rd !== 0 && s.ex_mem.rd === rs) { ev.push(`EX/MEM->${which}`); return s.ex_mem.alu; } if (s.mem_wb && s.mem_wb.c.RegWrite && s.mem_wb.rd !== 0 && s.mem_wb.rd === rs) { ev.push(`MEM/WB->${which}`); return s.mem_wb.val; } return null; }; const fa = fwd(e.rs1, e.useRs1, "a"), fb = fwd(e.rs2, e.useRs2, "b"); const a = fa === null ? e.a : fa; const bReg = fb === null ? e.b : fb; const b = e.c.ALUSrc ? e.imm : bReg; const alu = e.c.ALUOp === "SUB" ? (a - b) | 0 : e.c.ALUOp === "AND" ? a & b : e.c.ALUOp === "OR" ? a | b : (a + b) | 0; if (e.c.Branch && a === bReg) { taken = true; target = e.pc + e.imm; } ex_mem = { pc: e.pc, name: e.name, rd: e.rd, alu, storeVal: bReg, c: e.c }; } // -- Hazard detection: load-use -> stall; taken branch -> flush. -- const d = s.if_id ? decode(s.if_id.word) : null; const loadUse = !!(d && s.id_ex && s.id_ex.c.MemRead && s.id_ex.rd !== 0 && ((d.useRs1 && d.rs1 === s.id_ex.rd) || (d.useRs2 && d.rs2 === s.id_ex.rd))); if (taken) { ev.push(`BEQ taken -> flush 2, pc=${target}`); return { pc: target, regs, mem, if_id: null, id_ex: null, ex_mem, mem_wb }; } if (loadUse) { ev.push("load-use -> stall (bubble into EX)"); return { pc: s.pc, regs, mem, if_id: s.if_id, id_ex: null, ex_mem, mem_wb }; } // -- ID and IF as usual. -- let id_ex: IDEX | null = null; if (d && s.if_id) id_ex = { pc: s.if_id.pc, name: d.name, rd: d.rd, rs1: d.rs1, rs2: d.rs2, useRs1: d.useRs1, useRs2: d.useRs2, a: regs[d.rs1], b: regs[d.rs2], imm: d.imm, c: control(d) }; const word = imem[s.pc >> 2]; const if_id: IFID | null = word === undefined ? null : { pc: s.pc, word }; return { pc: s.pc + 4, regs, mem, if_id, id_ex, ex_mem, mem_wb }; } // -- The dependent program that broke lesson 38 -- plus a store, a load-use, and a taken branch. -- const imem = [ asm.ADDI(1, 0, 5), // I0 0: x1 = 5 asm.ADDI(2, 0, 7), // I1 4: x2 = 7 asm.ADD(3, 1, 2), // I2 8: x3 = x1 + x2 <- RAW on x1 AND x2 asm.SW(3, 0, 0), // I3 12: mem[0] = x3 <- RAW on x3 asm.LW(4, 0, 0), // I4 16: x4 = mem[0] asm.ADD(5, 4, 3), // I5 20: x5 = x4 + x3 <- load-use on x4! asm.BEQ(5, 5, 12), // I6 24: if x5 == x5 goto 36 (always taken) asm.ADDI(6, 0, 99), // I7 28: x6 = 99 -- must be squashed asm.ADDI(6, 0, 88), // I8 32: x6 = 88 -- must be squashed asm.ADDI(7, 0, 1), // I9 36: x7 = 1 ]; const tag = (x: { pc: number } | null) => (x ? "I" + (x.pc >> 2) : "."); let s: State = { pc: 0, regs: new Array(32).fill(0), mem: new Array(16).fill(0), if_id: null, id_ex: null, ex_mem: null, mem_wb: null }; console.log("cycle | IF ID EX MEM WB | events"); console.log("------+---------------------+-------"); for (let cycle = 0; cycle < 40; cycle++) { const fetching = imem[s.pc >> 2] !== undefined ? { pc: s.pc } : null; const row = [fetching, s.if_id, s.id_ex, s.ex_mem, s.mem_wb].map((x) => tag(x as { pc: number } | null).padStart(3)).join(" "); if (!fetching && !s.if_id && !s.id_ex && !s.ex_mem && !s.mem_wb) { console.log(` ${String(cycle).padStart(2)} | ${row} |`); break; } const ev: string[] = []; s = next(s, imem, ev); console.log(` ${String(cycle).padStart(2)} | ${row} | ${ev.join(", ")}`); } console.log(`\nx1=${s.regs[1]} x2=${s.regs[2]} x3=${s.regs[3]} x4=${s.regs[4]} x5=${s.regs[5]} x6=${s.regs[6]} x7=${s.regs[7]} mem[0]=${s.mem[0]}`); console.log("expected: x1=5 x2=7 x3=12 x4=12 x5=24 x6=0 x7=1 mem[0]=12");

Sixteen cycles, and every register lands right: the RAW pair is healed by two simultaneous bypasses at cycle 4 (MEM/WB→a, EX/MEM→b — the exact arrows from the figure), the load-use pair costs precisely one bubble at cycle 6, and the taken BEQ at cycle 9 flushes I7 and I8 — x6 stays 0, untouched by the two squashed writes. Flip FORWARDING to false at the top and run it again: same program, wrong registers everywhere — a one-keystroke reminder of how load-bearing those comparators are.

The original MIPS said exactly that — the name stood for Microprocessor without Interlocked Pipeline Stages: no stall hardware, and a "delay slot" after every load and branch that the compiler had to fill (with a NOP if it found nothing better). It worked — until the pipeline changed. The delay slot was the implementation leaking into the contract: when later MIPS chips went superscalar and deeper, the one-slot assumption was wrong, yet every shipped binary encoded it, and the hardware had to fake the old pipeline forever. RISC-V, designed three decades wiser, has no delay slots — hazards are the hardware's private problem, which is why our forwarding unit and interlocks exist at all. It is the ISA-is-a-contract lesson with teeth: never let this year's microarchitecture write itself into the spec.