A SIMT Core in RTL
The contract is
signed; now we build the machine that honours it. A PrimerGPU core is the smallest
hardware embodiment of the SIMT bargain we studied in the
streaming
multiprocessor: one program counter, one fetch
unit and one decoder — feeding four lanes that execute every
instruction in lockstep. Four threads, one brain. The entire economy of a GPU is visible in that
ratio: the expensive thinking machinery (what should happen next?) is bought once and amortised,
while the cheap doing machinery (ALUs, registers) is stamped out per thread. This lesson wires it
up — a control FSM, a shared front end, and four almost-identical lane columns whose one
difference turns out to be everything.
The six-state heartbeat
Our core is a multi-cycle machine: each instruction walks through six FSM states,
one per clock. This is deliberately humbler than a pipeline — states, not stages — because the
capstone's goal is a machine you can watch think:
\text{FETCH} \to \text{DECODE} \to \text{REQUEST} \to \text{WAIT} \to \text{EXECUTE} \to \text{UPDATE} \to \text{FETCH} \to \cdots
- FETCH — read the instruction word at
PC from program memory;
- DECODE — split it into opcode and fields (pure combinational unpacking —
our boring encoding table pays off here);
- REQUEST — for
LDR/STR, each lane computes its own
address and issues a memory request;
- WAIT — hold until the memory answers. For now memory replies in one cycle;
in two lessons, when the memory system becomes real, this state is where cores will visibly
idle — the FSM already has the waiting room built;
- EXECUTE — all four ALUs compute, all at once, on their own lane's
registers;
- UPDATE — commit: write results into the lane register files, set the flags,
advance (or branch) the shared
PC.
In SystemVerilog the spine is a dozen lines:
typedef enum logic [2:0] {FETCH, DECODE, REQUEST, WAIT, EXECUTE, UPDATE} state_t;
state_t st;
always_ff @(posedge clk) begin
if (!rst_n) st <= FETCH;
else unique case (st)
FETCH: st <= DECODE;
DECODE: st <= REQUEST;
REQUEST: st <= WAIT;
WAIT: st <= mem_pending ? WAIT : EXECUTE; // the stall lives HERE
EXECUTE: st <= UPDATE;
UPDATE: st <= FETCH;
endcase
end
Every instruction visits every state — even a NOP strolls through the memory states
doing nothing. Wasteful? Slightly. Simple enough to trust completely? Absolutely — and when the
dispatcher and memory controller arrive in the next lessons, this predictable six-beat rhythm is
what will make whole-machine traces readable.
The floor plan
Here is the whole core as a block diagram — watch how little of it is "thinking" and how much is
"doing":
The four columns come from a generate loop — hardware's version of stamping out
copies — and the loop variable itself becomes each lane's identity:
for (genvar l = 0; l < LANES; l++) begin : lane
regfile rf (
.clk (clk),
.block_idx (block_idx), // R13: which block this core is running
.block_dim (BLOCK_DIM), // R14: threads per block
.thread_idx (4'(l)), // R15: the genvar IS the thread index!
.rd(rd), .rs(rs), .rt(rt), .we(reg_we), .wdata(lane_result[l]),
.rs_val(rs_val[l]), .rt_val(rt_val[l])
);
alu alu_i (.op(alu_op), .a(rs_val[l]), .b(rt_val[l]), .y(alu_y[l]));
lsu lsu_i (.addr(rs_val[l]), .wdata(rt_val[l]), .req(mem_req), ...);
end
Read the thread_idx line twice: the elaboration-time constant l is
baked into lane l's register file as the value its R15
forever returns. Silicon has no loop at runtime — there are simply four register files, physically
wired to answer "who are you?" four different ways. That single wiring difference is what makes
the shared instruction stream compute four different array indices, fetch four different elements,
and store four different sums. Everything else in the columns is copy-paste.
- shared: one PC, one fetch, one decode, one six-state control FSM —
the cost of "what next?" is paid once;
- per lane: ALU, 16-entry register file, LSU, N/Z/P flags — the cost of
"do it" is paid per thread;
- lanes differ only in the hard-wired values of
R13–R15;
- memory instructions issue per-lane requests in REQUEST and stall the whole core in WAIT
until every lane's answer is back.
Predication — and one honest shortcut
Control flow is where lockstep gets philosophically interesting. CMP executes in
every lane, and each lane keeps its own N/Z/P flags — lane 0 might record
"negative" while lane 3 records "positive", because they compared different data. So what should
BRnzp do when the lanes disagree? There is only one PC — the lanes
cannot go separate ways.
Real GPUs answer with divergence machinery: mask off the lanes that didn't take the branch, run
both paths one after the other, reconverge — the elaborate apparatus of
branch
divergence and reconvergence. Our toy takes a documented shortcut:
PrimerGPU branches when lane 0's predicate holds, and the other lanes follow.
This is honest for the kernels we run — vector add has no branches at all, and matmul's loop
condition depends only on the loop counter, which is uniform (identical in every lane,
since it never touches R15) — so lane 0 speaks for everyone. But be clear about what
we gave up: a kernel that branches on per-thread data would silently do the wrong thing
in lanes 1–3. Real divergence support is the single biggest gap between our core and a real SM,
and it is first on the upgrade list in the final lesson's gap audit.
The core in the simulator
Now the payoff for learning the two-phase pattern back in the
sequential-logic
lesson: the whole core is one State object and one
next() function — settle, then snap. State is the FSM state, the PC, the instruction
register, four lanes of registers, and memory. Below, one core runs the lesson-72 vector-add
kernel (rebased for 4 elements: a@0, b@4, c@8) as one
4-thread block. Watch the FSM tick through its six beats, and watch the lane brackets — four
different values from one instruction, every time:
// ── One SIMT core: ONE pc + FSM, FOUR lanes marching in lockstep. ──
// Vector add over 4 elements (one block): a@0, b@4, c@8. Ideal 1-cycle memory for now.
const PROG = [0x50de, 0x300f, 0x9100, 0x9204, 0x9308, 0x3410, 0x7440, 0x3520, 0x7550, 0x3645, 0x3730, 0x8076, 0xf000];
const NAMES = ["NOP", "BRnzp", "CMP", "ADD", "SUB", "MUL", "DIV", "LDR", "STR", "CONST", "?", "?", "?", "?", "?", "RET"];
type Fsm = "FETCH" | "DECODE" | "REQUEST" | "WAIT" | "EXECUTE" | "UPDATE";
const STEP: Record<Fsm, Fsm> = {
FETCH: "DECODE", DECODE: "REQUEST", REQUEST: "WAIT", WAIT: "EXECUTE", EXECUTE: "UPDATE", UPDATE: "FETCH",
};
interface Lane { regs: number[]; nzp: number; addr: number; data: number }
interface State { st: Fsm; pc: number; ir: number; done: boolean; lanes: Lane[]; mem: number[] }
function makeState(blockIdx: number, blockDim: number, mem: number[]): State {
const lanes: Lane[] = [];
for (let t = 0; t < 4; t++) {
const regs = new Array(16).fill(0);
regs[13] = blockIdx; regs[14] = blockDim; regs[15] = t; // each lane's ONLY difference
lanes.push({ regs, nzp: 0, addr: 0, data: 0 });
}
return { st: "FETCH", pc: 0, ir: 0, done: false, lanes, mem };
}
// The combinational logic: old state in, next state out. Settle, then snap.
function next(s: State): State {
const n: State = {
...s, st: STEP[s.st], mem: [...s.mem],
lanes: s.lanes.map((l) => ({ ...l, regs: [...l.regs] })),
};
const w = s.ir, op = w >> 12, rd = (w >> 8) & 15, rs = (w >> 4) & 15, rt = w & 15, imm = w & 255;
if (s.st === "FETCH") n.ir = PROG[s.pc];
if (s.st === "REQUEST" && (op === 7 || op === 8))
n.lanes.forEach((l) => { l.addr = l.regs[rs]; }); // each lane computes ITS address
if (s.st === "WAIT") {
if (op === 7) n.lanes.forEach((l) => { l.data = s.mem[l.addr]; }); // ideal memory: LDR data back
if (op === 8) n.lanes.forEach((l) => { n.mem[l.addr] = l.regs[rt]; }); // STR performs the write
}
if (s.st === "EXECUTE") n.lanes.forEach((l) => {
const a = l.regs[rs], b = l.regs[rt];
if (op === 3) l.data = a + b;
if (op === 4) l.data = a - b;
if (op === 5) l.data = a * b;
if (op === 6) l.data = Math.floor(a / b);
if (op === 9) l.data = imm;
if (op === 2) l.nzp = a - b < 0 ? 4 : a - b === 0 ? 2 : 1;
});
if (s.st === "UPDATE") {
if ((op >= 3 && op <= 7) || op === 9)
n.lanes.forEach((l) => { if (rd < 13) l.regs[rd] = l.data; }); // R13–R15 stay read-only
if (op === 15) n.done = true;
n.pc = op === 1 && (s.lanes[0].nzp & (rd >> 1)) !== 0 ? imm : s.pc + 1; // branch: lane 0 decides (toy!)
}
return n;
}
// ── Run one block (blockIdx = 0, blockDim = 4) and watch the FSM march. ──
const mem = [10, 20, 30, 40, 1, 2, 3, 4, 0, 0, 0, 0]; // a | b | c
let s = makeState(0, 4, mem);
s.lanes.forEach((l, t) => console.log(`lane ${t}: %blockIdx=${l.regs[13]} %blockDim=${l.regs[14]} %threadIdx=${l.regs[15]}`));
console.log("");
const dis = (w: number) => {
const op = w >> 12, rd = (w >> 8) & 15, rs = (w >> 4) & 15, rt = w & 15, imm = w & 255;
if (op === 15 || op === 0) return NAMES[op];
if (op === 9) return `CONST R${rd}, #${imm}`;
if (op === 7) return `LDR R${rd}, R${rs}`;
if (op === 8) return `STR R${rs}, R${rt}`;
return `${NAMES[op]} R${rd}, R${rs}, R${rt}`;
};
let cycle = 0;
while (!s.done) {
const label = s.st === "FETCH" ? `pc=${s.pc}` : dis(s.ir);
let line = `c${String(cycle).padStart(2, "0")} ${s.st.padEnd(7)} ${label}`;
const nx = next(s);
if (s.st === "UPDATE") { // show what the edge committed, per lane
const rd = (s.ir >> 8) & 15, op = s.ir >> 12;
if (((op >= 3 && op <= 7) || op === 9) && rd < 13)
line += ` -> R${rd} = [${nx.lanes.map((l) => l.regs[rd]).join(", ")}]`;
if (op === 8) line += ` -> mem = [${nx.mem.join(", ")}]`;
}
console.log(line);
s = nx;
cycle++;
}
console.log(`\ndone in ${cycle} cycles (13 instructions x 6 FSM states)`);
console.log(`c = [${s.mem.slice(8).join(", ")}] (expected 11, 22, 33, 44)`);
Read the trace like a hardware engineer. Cycle 11's ADD R0, R0, R15 commits
R0 = [0, 1, 2, 3] — four different indices from one instruction, the index magic
happening live. Cycle 41's LDR returns [10, 20, 30, 40] — four
different memory words from one opcode. And the whole 13-instruction kernel lands on exactly
13 \times 6 = 78 cycles with c = [11, 22, 33, 44].
One core down. Next lesson: a dispatcher that feeds blocks to several of them.
The classic RISC pipeline's five stages and our six states look like cousins,
but they are different animals. A pipeline has all five stages busy at once — five instructions
in flight, one finishing per cycle; our multi-cycle FSM works on one instruction at a time, six
cycles each. So is PrimerGPU's core just a bad CPU? Per thread, yes — gloriously bad.
But look at what each cycle accomplishes: EXECUTE fires four ALUs, REQUEST issues
four memory addresses. The GPU wager, visible even at this toy scale, is that width
beats depth: rather than heroically overlapping one thread's instructions, just do four threads
plainly — then, in the coming lessons, eight, twenty-four, thousands. Real GPU cores eventually
pipeline too (why not both?), but their souls remain wide-and-simple, and per-thread performance
remains the metric GPUs cheerfully lose.
A tempting "simplification": since all lanes run the same instruction, let them share one
register file and save three-quarters of the flip-flops. This breaks on the first
instruction, not eventually — because in SIMT, every instruction writes the same register
number with a different value in each lane. When
ADD R0, R0, %threadIdx commits, lane 0 must hold R0 = 0 while lane 3
holds R0 = 3; with one shared R0, the last writer wins and all four
threads become thread 3 — vector add fills c with four copies of
a[3]+b[3]. "Same register name" in SIMT means same role,
never same storage: 4 \times 16 = 64 physical registers, no
negotiation. Scaled up, this is why real SMs carry those outlandish 256 KB register files —
thousands of threads, each owning its private copy of "R0".