The pipeline has an appetite: it wants to fetch a new instruction every single cycle, no exceptions. That works beautifully for a straight run of code — the next instruction is always at the next address. But roughly one instruction in six is a branch, and a branch asks an awkward question: which instruction comes next? If the branch is taken, the next one is far away at the target; if not, it's the very next line. And the pipeline doesn't find out the answer until several cycles after it has already, greedily, fetched something.
That is a control hazard: the machine must fetch before it knows what to fetch. Whatever
it guessed while waiting may be wrong, and wrong instructions already climbing the pipeline must be thrown
away. This lesson is about the size of that bill — the branch penalty — and the four
classic ways architects fight it. It sets the stage for the
In the classic five-stage pipeline a conditional branch does not know its outcome or its target until it has computed the condition and the target address — around the MEM stage, in cycle 4 for a branch fetched in cycle 1. But in cycles 2, 3 and 4 the fetch unit has already grabbed three more instructions from the fall-through path. If the branch turns out to be taken, all three are wrong. They must be flushed — turned into bubbles — and fetch restarted at the real target. That is a three-cycle penalty on every taken branch.
Step through the figure. The branch (beq) climbs the pipe; behind it, three fall-through
instructions are fetched on speculation; when the branch resolves in MEM and turns out taken, those three
are crossed out and fetch is redirected to the real target — three cycles of work in the bin.
Three lost cycles doesn't sound catastrophic until you multiply. If branches are
Architects have four classic weapons, from crudest to cleverest:
| Technique | Idea | Cost |
|---|---|---|
| Stall (freeze) | fetch nothing until the branch resolves | the full penalty on every branch — simplest, slowest |
| Flush (predict-not-taken) | assume not-taken, keep fetching fall-through; squash if wrong | penalty only on taken branches — free when not taken |
| Delayed branch | ISA exposes a "branch delay slot" the compiler fills with useful work | zero penalty if a useful instruction fits the slot |
| Resolve earlier | move condition + target compute into ID | shrinks the penalty itself (3 → 1) |
The last one is the key structural insight: the penalty is how late the branch resolves, so add a little extra comparator and an adder to the ID stage and resolve the branch there, in cycle 2. Now only one instruction (the one fetched in cycle 2) is ever wrong, and the penalty drops from three to one. MIPS did exactly this. Every later trick — dynamic prediction, speculation — chases the same goal: never waste a fetch.
Say
so the pipeline runs at
Early MIPS and SPARC baked a delay slot into the ISA: the instruction right after a branch always executes, taken or not, so the compiler could hide one cycle of penalty by moving useful work into it. Clever — but it welded a specific pipeline depth into the permanent, programmer-visible contract. When pipelines grew deeper and gained real branch predictors, one delay slot was both too few to matter and an eternal nuisance the hardware had to keep honouring for backward compatibility. It is a textbook case of leaking a microarchitectural detail into the ISA. RISC-V, learning the lesson, has no delay slot — it trusts the branch predictor instead.
Predict-not-taken is a hardware convenience, not a claim about program behaviour. It's cheap because "keep fetching the next sequential instruction" is what the fetch unit does anyway — you only pay when you're wrong. But loops branch backwards and are usually taken, so a naive not-taken guess is often wrong exactly where it matters most. That gap is precisely why real processors moved to dynamic branch prediction, which learns each branch's habit and reaches 95%+ accuracy. Don't read "predict-not-taken" as a statement that branches are rarely taken — it's a statement about which guess is free to make.