Superscalar Execution
A perfectly pipelined machine has a hard ceiling: it finishes at most one instruction per
cycle. Every stage is busy, every cycle produces one result — and that's the best a single
pipeline can ever do, a \text{CPI} = 1. To go faster you must break the "one"
barrier and retire several instructions in the same cycle. A processor that fetches, issues, and
completes more than one instruction per clock is called superscalar, and
it is how every mainstream CPU since the mid-1990s has beaten the one-per-cycle wall.
The bookkeeping metric flips. Instead of \text{CPI} \ge 1 we now talk about
IPC — instructions per cycle — and a good superscalar core runs at
\text{IPC} > 1, equivalently \text{CPI} < 1. An
Apple or AMD core today can sustain around \text{IPC} \approx 4–6
on friendly code. The width at which a machine can fetch and issue is its issue width — a
4-wide machine can start up to four instructions per cycle.
IPC and CPI are reciprocals
The two numbers describe the same throughput from opposite ends:
\text{IPC} \;=\; \frac{\text{instructions}}{\text{cycles}}, \qquad \text{CPI} \;=\; \frac{\text{cycles}}{\text{instructions}} \;=\; \frac{1}{\text{IPC}}.
Plug IPC straight into the
iron law: since
\text{CPU time} = \text{IC} \times \text{CPI} \times T_{\text{cycle}}, doubling
IPC (halving CPI) at the same clock halves the running time. Superscalar execution is a direct assault on
the CPI term — the one owned by the microarchitecture.
- a superscalar processor fetches, issues and completes several instructions per
cycle, so \text{IPC} > 1 and \text{CPI} < 1;
- its issue width w is the most instructions it can start
in one cycle;
- it needs multiple execution ports (duplicated functional units) so several
instructions can execute at once;
- real IPC is far below w because of dependences, branches and memory
stalls — width is an upper bound, not a promise.
Multiple ports: many hands
You cannot issue four instructions per cycle into a single adder. A superscalar core duplicates its
execution resources into ports, each a lane to a functional unit. A typical core might
have several integer ALUs, one or two load/store ports, a couple of floating-point/SIMD units, and a
branch unit — the mix matters, because if a cycle's four ready instructions are all divides and
there's one divider, three of them wait. The scheduler's job is to pack ready instructions onto free
ports every cycle, and it leans entirely on the
register renaming
and reservation stations from the previous lesson to find enough independent work.
// How much does a superscalar core actually gain? IPC is capped by BOTH the issue
// width AND the available parallelism (here, a fraction of instructions are dependent
// on the immediately preceding one, so they cannot co-issue).
function ipc(width: number, dependentFraction: number): number {
// Independent instructions can fill ports up to 'width'; dependent ones serialise.
// A simple model: effective parallelism = 1 / dependentFraction, capped by width.
const parallelism = dependentFraction <= 0 ? width : Math.min(width, 1 / dependentFraction);
return parallelism;
}
const dep = 0.3; // 30% of instructions depend on the one just before them
console.log(`dependent fraction = ${dep}`);
for (const w of [1, 2, 4, 8, 16]) {
console.log(`issue width ${String(w).padStart(2)} -> IPC ~= ${ipc(w, dep).toFixed(2)} (CPI ~= ${(1 / ipc(w, dep)).toFixed(3)})`);
}
console.log("Note how IPC stops climbing once width outruns the available parallelism.");
Run it: widening from 1 to 4 helps a lot, but 8 and 16 barely move the needle — the code simply doesn't
contain enough independent instructions to fill the extra ports. That diminishing return is the whole story
of the limits of ILP.
The catch: dependency checking is quadratic
Issuing w instructions together is not free. Before they can go, the hardware
must check every pair within the group for dependences: does instruction 3's source match
instruction 1's destination? Instruction 2's? With w instructions there are
about \binom{w}{2} = \tfrac{w(w-1)}{2} pairs, and each source operand must be
compared against every earlier destination — so the number of comparators scales as
\text{comparisons} \;\propto\; w^2.
Double the width and the dependency-check logic roughly quadruples. The wakeup/select
logic and the register-file ports grow super-linearly too. This is why nobody ships a 32-wide core: the
checking hardware becomes slower (lengthening the cycle time — the very term you were trying to protect)
and hungrier for power, while the IPC payoff keeps shrinking. Watch the curve bend upward:
Worked example: width vs reality
Suppose a 4-wide core runs a loop of 10^9 instructions and sustains an average
\text{IPC} = 2.5 (well below its width of 4, because of dependences and
branches). Its CPI is 1/2.5 = 0.4. At 3\ \text{GHz}:
\text{CPU time} = \text{IC} \times \text{CPI} \times T_{\text{cycle}} = 10^9 \times 0.4 \times \tfrac{1}{3\times10^9}\ \text{s} \approx 0.133\ \text{s}.
The single-issue version (\text{IPC} = 1) would take
0.333\ \text{s} — a 2.5\times speedup, exactly the
IPC gain. The machine is 4-wide but delivers 2.5; that gap between peak width and
sustained IPC is where all the interesting architecture lives.
They both retire many instructions per cycle, but they disagree on who does the scheduling. A
superscalar core figures out at run time, in hardware, which instructions are
independent and which ports are free — every cycle, dynamically. A
VLIW machine pushes that job onto the compiler, which
bundles independent operations into one wide instruction ahead of time; the hardware just fires the slots.
Superscalar spends transistors (the quadratic checkers) to adapt to whatever code arrives; VLIW spends
compile-time cleverness and hopes the schedule holds up at run time. History mostly favoured superscalar —
for reasons we'll dig into in the VLIW lesson.
A "6-wide" core does not run at \text{IPC} = 6. Width is the maximum
number of instructions it can start in a cycle; the number it actually starts is limited
by true (RAW) dependences, branch mispredictions, cache misses, and port contention. Real workloads
rarely sustain much above \text{IPC} \approx 2–4 even
on very wide machines. Quoting the width as the throughput is a classic spec-sheet trap — always ask for
sustained IPC on a real benchmark, not the peak issue width.
Why 4-ish is the sweet spot
Put the two forces together. Widening raises the ceiling on IPC, but real code rarely has enough
independent instructions to reach it, so the extra ports sit idle — while the dependency-check and wakeup
logic cost grows as w^2, eating power and threatening the clock. The curves
cross around w \approx 4–6 for general-purpose code,
which is why mainstream cores clustered there for two decades. To go faster the industry stopped widening
one core and started replicating whole cores — the pivot to
multicore that the last lesson of this module sets
up.