Running a Kernel End to End
Four lessons of parts; now the machine. This is the payoff lesson — the one where an ISA on
paper, a lockstep core, a dealing dispatcher and a bottlenecked
memory
controller click together and PrimerGPU breathes: a kernel goes in one end
and eight correct sums come out the other, with every one of the 93 cycles in between accounted
for. We will watch the whole life of a launch — dispatch, lockstep fetch, per-lane index math,
coalesced loads queueing, the honest stalls, stores, RET, done, kernel-complete —
then interrogate the run like engineers: where did the cycles go? and what happens
if we take pieces away? Three configurations, one truth about GPUs, and a self-check in the
proper testbench
tradition to prove none of it is wishful thinking.
The complete machine
Every box on this diagram is something you built: the
contract
it all serves, the cores from lesson two, the dispatcher from lesson three, the memory system
from lesson four. The simulator below contains all of it in about 150 lines — dispatcher, both
cores, queues, arbitration, coalescer — still the same two-phase
settle-then-snap loop we started with back in the RTL module, just with a grander
State.
THE run
The launch: vector add over 8 elements as 2 blocks of 4 threads, on 2 cores with the coalescer
on. One line per cycle — each core's FSM state, each channel's countdown, and annotations when
something happens. Then the accounting, and then the same kernel with pieces removed:
// ── The whole PrimerGPU: dispatcher + 2 SIMT cores + memory controller + coalescer. ──
// Kernel: vector add, 8 elements as 2 blocks of 4 threads. a@0, b@8, c@16.
const PROG = [0x50de, 0x300f, 0x9100, 0x9208, 0x9310, 0x3410, 0x7440, 0x3520, 0x7550, 0x3645, 0x3730, 0x8076, 0xf000];
const NUM_BLOCKS = 2, LATENCY = 4;
const chanOf = (a: number) => Math.floor(a / 4) % 2;
type Fsm = "FETCH" | "DECODE" | "REQUEST" | "WAIT" | "EXECUTE" | "UPDATE" | "IDLE" | "DONE";
const STEP: Record<string, Fsm> = {
FETCH: "DECODE", DECODE: "REQUEST", REQUEST: "WAIT", WAIT: "EXECUTE", EXECUTE: "UPDATE", UPDATE: "FETCH",
};
interface Lane { regs: number[]; data: number }
interface Core { st: Fsm; pc: number; ir: number; pending: number; blockIdx: number; lanes: Lane[] }
interface Req { kind: "load" | "store"; addr: number; core: number; lane: number; data: number[] }
interface Chan { queue: Req[]; cur: Req | null; left: number }
interface M { cores: Core[]; chans: Chan[]; mem: number[]; nextBlock: number; doneBlocks: number; txns: number }
const idle = (): Core => ({ st: "IDLE", pc: 0, ir: 0, pending: 0, blockIdx: -1, lanes: [] });
const start = (b: number): Core => ({
st: "FETCH", pc: 0, ir: 0, pending: 0, blockIdx: b,
lanes: [0, 1, 2, 3].map((t) => {
const regs = new Array(16).fill(0);
regs[13] = b; regs[14] = 4; regs[15] = t;
return { regs, data: 0 };
}),
});
function coreNext(s: Core, ci: number, out: Req[], coalesce: boolean): Core {
if (s.st === "IDLE" || s.st === "DONE") return s;
const n: Core = { ...s, st: STEP[s.st], lanes: s.lanes.map((l) => ({ ...l, regs: [...l.regs] })) };
const w = s.ir, op = w >> 12, rd = (w >> 8) & 15, rs = (w >> 4) & 15, rt = w & 15, imm = w & 255;
if (s.st === "FETCH") n.ir = PROG[s.pc];
if (s.st === "REQUEST" && (op === 7 || op === 8)) {
const kind = op === 7 ? "load" as const : "store" as const;
const addrs = s.lanes.map((l) => l.regs[rs]);
if (coalesce && addrs.every((a, t) => a === addrs[0] + t)) {
out.push({ kind, addr: addrs[0], core: ci, lane: -1, data: s.lanes.map((l) => l.regs[rt]) });
n.pending = 1;
} else {
s.lanes.forEach((l, t) => out.push({ kind, addr: addrs[t], core: ci, lane: t, data: [l.regs[rt]] }));
n.pending = 4;
}
}
if (s.st === "WAIT" && s.pending > 0) n.st = "WAIT";
if (s.st === "EXECUTE") n.lanes.forEach((l) => {
const a = l.regs[rs], b = l.regs[rt];
l.data = op === 3 ? a + b : op === 4 ? a - b : op === 5 ? a * b : op === 6 ? Math.floor(a / b) : op === 9 ? imm : l.data;
});
if (s.st === "UPDATE") {
if ((op >= 3 && op <= 7) || op === 9) n.lanes.forEach((l) => { if (rd < 13) l.regs[rd] = l.data; });
if (op === 15) n.st = "DONE";
n.pc = s.pc + 1;
}
return n;
}
function machineNext(m: M, rr: number, coalesce: boolean, note: (s: string) => void): M {
const mem = [...m.mem];
const out: Req[] = [];
const cores = m.cores.map((c, i) => coreNext(c, i, out, coalesce));
const chans = m.chans.map((ch) => ({ ...ch, queue: [...ch.queue] }));
let { nextBlock, doneBlocks, txns } = m;
// memory channels: grind, deliver by requestor ID
chans.forEach((ch, k) => {
if (ch.cur) {
ch.left--;
if (ch.left === 0) {
const r = ch.cur, c = cores[r.core];
if (r.kind === "load") {
if (r.lane < 0) c.lanes.forEach((l, t) => { l.data = mem[r.addr + t]; });
else c.lanes[r.lane].data = mem[r.addr];
} else if (r.lane < 0) r.data.forEach((d, t) => { mem[r.addr + t] = d; });
else mem[r.addr] = r.data[0];
c.pending--;
note(`ch${k} answers core ${r.core}${r.lane < 0 ? " (all 4 lanes)" : " lane " + r.lane}`);
ch.cur = null;
}
}
});
// arbitration: rotating requestor priority into the channel queues
const rid = (r: Req) => r.core * 4 + Math.max(r.lane, 0);
out.sort((x, y) => ((rid(x) - rr + 8) % 8) - ((rid(y) - rr + 8) % 8));
out.forEach((r) => { chans[chanOf(r.addr)].queue.push(r); txns++; });
chans.forEach((ch) => { if (!ch.cur && ch.queue.length > 0) { ch.cur = ch.queue.shift()!; ch.left = LATENCY; } });
// dispatcher: collect done, grant one block per cycle to the lowest free core
m.cores.forEach((c, i) => {
if (c.st === "DONE") { doneBlocks++; cores[i] = idle(); note(`core ${i} done (block ${c.blockIdx}) — ${doneBlocks}/${NUM_BLOCKS}`); }
});
const free = cores.findIndex((c) => c.st === "IDLE");
if (nextBlock < NUM_BLOCKS && free >= 0) {
cores[free] = start(nextBlock);
note(`dispatch block ${nextBlock} -> core ${free}`);
nextBlock++;
}
return { cores, chans, mem, nextBlock, doneBlocks, txns };
}
function run(numCores: number, coalesce: boolean, verbose: boolean) {
const mem = new Array(24).fill(0);
for (let i = 0; i < 8; i++) { mem[i] = 10 * (i + 1); mem[8 + i] = i + 1; }
let m: M = {
cores: Array.from({ length: numCores }, idle),
chans: [{ queue: [], cur: null, left: 0 }, { queue: [], cur: null, left: 0 }],
mem, nextBlock: 0, doneBlocks: 0, txns: 0,
};
const budget: Record<string, number> = {};
let cycle = 0, waitCycles = 0;
while (m.doneBlocks < NUM_BLOCKS) {
const notes: string[] = [];
const snap = m;
m = machineNext(m, cycle % 8, coalesce, (s) => notes.push(s));
snap.cores.forEach((c) => {
budget[c.st] = (budget[c.st] ?? 0) + 1;
if (c.st === "WAIT") waitCycles++;
});
if (verbose) {
const cs = snap.cores.map((c, i) => `core${i} ${(c.st === "FETCH" ? "FETCH p" + c.pc : c.st).padEnd(8)}`).join(" ");
const ch = snap.chans.map((c, k) => `ch${k} ${c.cur ? "busy" + c.left : "- "}${c.queue.length > 0 ? "+q" + c.queue.length : " "}`).join(" ");
console.log(`c${String(cycle).padStart(2, "0")} ${cs} ${ch} ${notes.join("; ")}`);
}
cycle++;
}
// self-check: the RTL's memory must equal a plain software reference (the testbench discipline)
const ref = new Array(24).fill(0);
for (let i = 0; i < 8; i++) { ref[i] = 10 * (i + 1); ref[8 + i] = i + 1; ref[16 + i] = ref[i] + ref[8 + i]; }
const pass = m.mem.every((x, i) => x === ref[i]);
return { cycle, txns: m.txns, budget, waitCycles, pass, c: m.mem.slice(16) };
}
// ── THE run: the full machine, watched cycle by cycle. ──
console.log("PrimerGPU end to end: 2 cores, coalescer ON, 2 blocks of 4 threads");
const A = run(2, true, true);
console.log(`\nkernel complete at cycle ${A.cycle}; c = [${A.c.join(", ")}]; self-check ${A.pass ? "PASS" : "FAIL"}`);
console.log("\nwhere the core-cycles went (both cores, config A):");
const total = Object.values(A.budget).reduce((a, b) => a + b, 0);
(["FETCH", "DECODE", "REQUEST", "WAIT", "EXECUTE", "UPDATE", "IDLE"] as const).forEach((st) => {
const n = A.budget[st] ?? 0;
console.log(` ${st.padEnd(7)} ${String(n).padStart(4)} cycles ${String(Math.round((100 * n) / total)).padStart(3)}% ${"#".repeat(Math.round(n / 4))}`);
});
console.log("\nthree configurations, same kernel, same data:");
const B = run(1, true, false);
const C = run(2, false, false);
console.log(` A: 2 cores, coalescer ON : ${A.cycle} cycles, ${A.txns} transactions (self-check ${A.pass ? "PASS" : "FAIL"})`);
console.log(` B: 1 core, coalescer ON : ${B.cycle} cycles, ${B.txns} transactions (self-check ${B.pass ? "PASS" : "FAIL"})`);
console.log(` C: 2 cores, coalescer OFF: ${C.cycle} cycles, ${C.txns} transactions (self-check ${C.pass ? "PASS" : "FAIL"})`);
console.log(` parallelism (B/A): ${(B.cycle / A.cycle).toFixed(2)}x coalescing (C/A): ${(C.cycle / A.cycle).toFixed(2)}x`);
Now walk the trace like a guided tour — every phase of a kernel's life is in there:
- c00–c01, the launch: the dispatcher grants block 0 to core 0, then block 1
to core 1 — one grant per cycle,
%blockIdx latched at each;
- c02–c39, the lockstep march: both cores tick FETCH→…→UPDATE through the
index math and the base constants, core 1 forever one beat behind. At c07's UPDATE, eight
threads across two cores have each computed a different i;
- c40–c44, memory gets real: both cores' first coalesced loads hit the
channels — block 0's line lands on ch0, block 1's on ch1, served in parallel — while
both cores sit visibly in WAIT;
- c56–c60 and c78–c82: the second load and the store replay the pattern —
three memory instructions, three stalls, exactly as the
memory
lesson measured;
- c91–c93, the curtain: each core's
RET raises done, the
dispatcher counts 1/2 then 2/2, and kernel-complete fires at cycle 93 with
c = [11, 22, \dots, 88].
Where the cycles went
The budget table is the honest heart of this lesson:
| State | Core-cycles | Share | What it is |
| FETCH | 26 | 14% | reading instruction words |
| DECODE | 26 | 14% | unpacking fields |
| REQUEST | 26 | 14% | computing addresses, issuing requests |
| WAIT | 50 | 27% | stalled on the memory system |
| EXECUTE | 26 | 14% | the ALUs actually computing |
| UPDATE | 26 | 14% | committing registers and the PC |
| IDLE | 4 | 2% | launch and drain ramps |
WAIT is the largest line — nearly double any other state — and that is
with the coalescer doing its best; switch it off (config C) and waiting swells to
roughly half of all core time. Meanwhile EXECUTE, the only state where mathematics actually
happens, gets one cycle in seven. Our toy, built honestly, has independently discovered the
central truth of GPU design: a GPU is a machine for surviving memory latency,
and everything real GPUs add — warps in flight, caches, deeper coalescing, occupancy machinery —
is aimed squarely at that 27%.
Three machines, one kernel
The comparison table quantifies the two big architectural ideas on our own machine,
not by appeal to authority:
| Config | Cores | Coalescer | Cycles | Transactions | vs. A |
| A | 2 | on | 93 | 6 | — |
| B | 1 | on | 183 | 6 | 1.97× slower |
| C | 2 | off | 129 | 24 | 1.39× slower |
Halve the cores and the kernel takes 1.97× longer — block-level dispatch converting hardware
directly into speed, the scalability contract paying out at
N{=}2. Keep both cores but scatter the memory traffic into per-lane
transactions and you lose 1.39× while moving the same 24 words. And in every
configuration, the self-check passes: the final data memory is compared,
word for word, against a reference computed by a plain software loop — the testbench discipline
from the verification module, in miniature. A trace that looks right is an anecdote; a
machine-checked equality against an independent model is evidence. The crown jewel is not that
the trace is pretty — it is that line: self-check PASS.
- the full pipeline works: launch → dispatch → lockstep execution → coalesced memory
traffic → stores → done → kernel-complete, in 93 cycles;
- WAIT dominates the cycle budget (27% coalesced, ~half uncoalesced) —
memory latency is the design problem;
- parallelism measured: 2 cores ⇒ 1.97×; coalescing measured:
1.39× on this kernel;
- correctness is established by asserting final memory equals an independent
software reference, never by eyeballing.
Pause on what just happened, because it is the whole course in one breath. You wrote text —
thirteen assembly lines. A program you wrote (the assembler) turned it into sixteen-bit words. A
machine you specified at the register-transfer level — FSMs, latches, arbitration, all things
that synthesize to gates — executed those words, and a testbench you understand proved the
answer right. That is the same loop a thousand-engineer GPU team runs, and the same loop this
course's earlier capstone ran for a CPU; the difference is only scale and the number of zeros on
the budget. There is a marvellous circularity available here too: our simulator ran on your
computer's processor — silicon that began life as somebody's RTL, simulated on some earlier
processor, all the way back down a chain of machines building machines that bottoms out in the
hand-drawn masks of the 1970s. You are now, in a small but genuine way, part of that chain.
It would be easy to read the budget table as an indictment — our cores twiddle in WAIT over a
quarter of the time; surely a better design would fix that? Look closer: the fix is not cleverer
circuits, it is more threads. Our machine runs 8 threads, so when both blocks
stall on loads there is nothing else to switch to — the WAIT shows up as naked idle
time. A real SM keeps dozens of warps resident precisely so that another warp's arithmetic can
fill every one of those cycles: same latency, hidden behind other work. Real GPUs run
hundreds of thousands of threads not because the workload demands that many answers at
once, but because that is how many are needed to bury the memory stalls our toy displays
nakedly. In other words: PrimerGPU's inefficiency is the most honest thing about it — it
exhibits the disease (latency) with the medicine (massive multithreading) removed, which is
exactly why the medicine exists. The next lesson makes that upgrade path precise.