A SIMT Core in RTL

The contract is signed; now we build the machine that honours it. A PrimerGPU core is the smallest hardware embodiment of the SIMT bargain we studied in the streaming multiprocessor: one program counter, one fetch unit and one decoder — feeding four lanes that execute every instruction in lockstep. Four threads, one brain. The entire economy of a GPU is visible in that ratio: the expensive thinking machinery (what should happen next?) is bought once and amortised, while the cheap doing machinery (ALUs, registers) is stamped out per thread. This lesson wires it up — a control FSM, a shared front end, and four almost-identical lane columns whose one difference turns out to be everything.

The six-state heartbeat

Our core is a multi-cycle machine: each instruction walks through six FSM states, one per clock. This is deliberately humbler than a pipeline — states, not stages — because the capstone's goal is a machine you can watch think:

\text{FETCH} \to \text{DECODE} \to \text{REQUEST} \to \text{WAIT} \to \text{EXECUTE} \to \text{UPDATE} \to \text{FETCH} \to \cdots

In SystemVerilog the spine is a dozen lines:

typedef enum logic [2:0] {FETCH, DECODE, REQUEST, WAIT, EXECUTE, UPDATE} state_t; state_t st; always_ff @(posedge clk) begin if (!rst_n) st <= FETCH; else unique case (st) FETCH: st <= DECODE; DECODE: st <= REQUEST; REQUEST: st <= WAIT; WAIT: st <= mem_pending ? WAIT : EXECUTE; // the stall lives HERE EXECUTE: st <= UPDATE; UPDATE: st <= FETCH; endcase end

Every instruction visits every state — even a NOP strolls through the memory states doing nothing. Wasteful? Slightly. Simple enough to trust completely? Absolutely — and when the dispatcher and memory controller arrive in the next lessons, this predictable six-beat rhythm is what will make whole-machine traces readable.

The floor plan

Here is the whole core as a block diagram — watch how little of it is "thinking" and how much is "doing":

The four columns come from a generate loop — hardware's version of stamping out copies — and the loop variable itself becomes each lane's identity:

for (genvar l = 0; l < LANES; l++) begin : lane regfile rf ( .clk (clk), .block_idx (block_idx), // R13: which block this core is running .block_dim (BLOCK_DIM), // R14: threads per block .thread_idx (4'(l)), // R15: the genvar IS the thread index! .rd(rd), .rs(rs), .rt(rt), .we(reg_we), .wdata(lane_result[l]), .rs_val(rs_val[l]), .rt_val(rt_val[l]) ); alu alu_i (.op(alu_op), .a(rs_val[l]), .b(rt_val[l]), .y(alu_y[l])); lsu lsu_i (.addr(rs_val[l]), .wdata(rt_val[l]), .req(mem_req), ...); end

Read the thread_idx line twice: the elaboration-time constant l is baked into lane l's register file as the value its R15 forever returns. Silicon has no loop at runtime — there are simply four register files, physically wired to answer "who are you?" four different ways. That single wiring difference is what makes the shared instruction stream compute four different array indices, fetch four different elements, and store four different sums. Everything else in the columns is copy-paste.

Predication — and one honest shortcut

Control flow is where lockstep gets philosophically interesting. CMP executes in every lane, and each lane keeps its own N/Z/P flags — lane 0 might record "negative" while lane 3 records "positive", because they compared different data. So what should BRnzp do when the lanes disagree? There is only one PC — the lanes cannot go separate ways.

Real GPUs answer with divergence machinery: mask off the lanes that didn't take the branch, run both paths one after the other, reconverge — the elaborate apparatus of branch divergence and reconvergence. Our toy takes a documented shortcut: PrimerGPU branches when lane 0's predicate holds, and the other lanes follow. This is honest for the kernels we run — vector add has no branches at all, and matmul's loop condition depends only on the loop counter, which is uniform (identical in every lane, since it never touches R15) — so lane 0 speaks for everyone. But be clear about what we gave up: a kernel that branches on per-thread data would silently do the wrong thing in lanes 1–3. Real divergence support is the single biggest gap between our core and a real SM, and it is first on the upgrade list in the final lesson's gap audit.

The core in the simulator

Now the payoff for learning the two-phase pattern back in the sequential-logic lesson: the whole core is one State object and one next() function — settle, then snap. State is the FSM state, the PC, the instruction register, four lanes of registers, and memory. Below, one core runs the lesson-72 vector-add kernel (rebased for 4 elements: a@0, b@4, c@8) as one 4-thread block. Watch the FSM tick through its six beats, and watch the lane brackets — four different values from one instruction, every time:

// ── One SIMT core: ONE pc + FSM, FOUR lanes marching in lockstep. ── // Vector add over 4 elements (one block): a@0, b@4, c@8. Ideal 1-cycle memory for now. const PROG = [0x50de, 0x300f, 0x9100, 0x9204, 0x9308, 0x3410, 0x7440, 0x3520, 0x7550, 0x3645, 0x3730, 0x8076, 0xf000]; const NAMES = ["NOP", "BRnzp", "CMP", "ADD", "SUB", "MUL", "DIV", "LDR", "STR", "CONST", "?", "?", "?", "?", "?", "RET"]; type Fsm = "FETCH" | "DECODE" | "REQUEST" | "WAIT" | "EXECUTE" | "UPDATE"; const STEP: Record<Fsm, Fsm> = { FETCH: "DECODE", DECODE: "REQUEST", REQUEST: "WAIT", WAIT: "EXECUTE", EXECUTE: "UPDATE", UPDATE: "FETCH", }; interface Lane { regs: number[]; nzp: number; addr: number; data: number } interface State { st: Fsm; pc: number; ir: number; done: boolean; lanes: Lane[]; mem: number[] } function makeState(blockIdx: number, blockDim: number, mem: number[]): State { const lanes: Lane[] = []; for (let t = 0; t < 4; t++) { const regs = new Array(16).fill(0); regs[13] = blockIdx; regs[14] = blockDim; regs[15] = t; // each lane's ONLY difference lanes.push({ regs, nzp: 0, addr: 0, data: 0 }); } return { st: "FETCH", pc: 0, ir: 0, done: false, lanes, mem }; } // The combinational logic: old state in, next state out. Settle, then snap. function next(s: State): State { const n: State = { ...s, st: STEP[s.st], mem: [...s.mem], lanes: s.lanes.map((l) => ({ ...l, regs: [...l.regs] })), }; const w = s.ir, op = w >> 12, rd = (w >> 8) & 15, rs = (w >> 4) & 15, rt = w & 15, imm = w & 255; if (s.st === "FETCH") n.ir = PROG[s.pc]; if (s.st === "REQUEST" && (op === 7 || op === 8)) n.lanes.forEach((l) => { l.addr = l.regs[rs]; }); // each lane computes ITS address if (s.st === "WAIT") { if (op === 7) n.lanes.forEach((l) => { l.data = s.mem[l.addr]; }); // ideal memory: LDR data back if (op === 8) n.lanes.forEach((l) => { n.mem[l.addr] = l.regs[rt]; }); // STR performs the write } if (s.st === "EXECUTE") n.lanes.forEach((l) => { const a = l.regs[rs], b = l.regs[rt]; if (op === 3) l.data = a + b; if (op === 4) l.data = a - b; if (op === 5) l.data = a * b; if (op === 6) l.data = Math.floor(a / b); if (op === 9) l.data = imm; if (op === 2) l.nzp = a - b < 0 ? 4 : a - b === 0 ? 2 : 1; }); if (s.st === "UPDATE") { if ((op >= 3 && op <= 7) || op === 9) n.lanes.forEach((l) => { if (rd < 13) l.regs[rd] = l.data; }); // R13–R15 stay read-only if (op === 15) n.done = true; n.pc = op === 1 && (s.lanes[0].nzp & (rd >> 1)) !== 0 ? imm : s.pc + 1; // branch: lane 0 decides (toy!) } return n; } // ── Run one block (blockIdx = 0, blockDim = 4) and watch the FSM march. ── const mem = [10, 20, 30, 40, 1, 2, 3, 4, 0, 0, 0, 0]; // a | b | c let s = makeState(0, 4, mem); s.lanes.forEach((l, t) => console.log(`lane ${t}: %blockIdx=${l.regs[13]} %blockDim=${l.regs[14]} %threadIdx=${l.regs[15]}`)); console.log(""); const dis = (w: number) => { const op = w >> 12, rd = (w >> 8) & 15, rs = (w >> 4) & 15, rt = w & 15, imm = w & 255; if (op === 15 || op === 0) return NAMES[op]; if (op === 9) return `CONST R${rd}, #${imm}`; if (op === 7) return `LDR R${rd}, R${rs}`; if (op === 8) return `STR R${rs}, R${rt}`; return `${NAMES[op]} R${rd}, R${rs}, R${rt}`; }; let cycle = 0; while (!s.done) { const label = s.st === "FETCH" ? `pc=${s.pc}` : dis(s.ir); let line = `c${String(cycle).padStart(2, "0")} ${s.st.padEnd(7)} ${label}`; const nx = next(s); if (s.st === "UPDATE") { // show what the edge committed, per lane const rd = (s.ir >> 8) & 15, op = s.ir >> 12; if (((op >= 3 && op <= 7) || op === 9) && rd < 13) line += ` -> R${rd} = [${nx.lanes.map((l) => l.regs[rd]).join(", ")}]`; if (op === 8) line += ` -> mem = [${nx.mem.join(", ")}]`; } console.log(line); s = nx; cycle++; } console.log(`\ndone in ${cycle} cycles (13 instructions x 6 FSM states)`); console.log(`c = [${s.mem.slice(8).join(", ")}] (expected 11, 22, 33, 44)`);

Read the trace like a hardware engineer. Cycle 11's ADD R0, R0, R15 commits R0 = [0, 1, 2, 3] — four different indices from one instruction, the index magic happening live. Cycle 41's LDR returns [10, 20, 30, 40] — four different memory words from one opcode. And the whole 13-instruction kernel lands on exactly 13 \times 6 = 78 cycles with c = [11, 22, 33, 44]. One core down. Next lesson: a dispatcher that feeds blocks to several of them.

The classic RISC pipeline's five stages and our six states look like cousins, but they are different animals. A pipeline has all five stages busy at once — five instructions in flight, one finishing per cycle; our multi-cycle FSM works on one instruction at a time, six cycles each. So is PrimerGPU's core just a bad CPU? Per thread, yes — gloriously bad. But look at what each cycle accomplishes: EXECUTE fires four ALUs, REQUEST issues four memory addresses. The GPU wager, visible even at this toy scale, is that width beats depth: rather than heroically overlapping one thread's instructions, just do four threads plainly — then, in the coming lessons, eight, twenty-four, thousands. Real GPU cores eventually pipeline too (why not both?), but their souls remain wide-and-simple, and per-thread performance remains the metric GPUs cheerfully lose.

A tempting "simplification": since all lanes run the same instruction, let them share one register file and save three-quarters of the flip-flops. This breaks on the first instruction, not eventually — because in SIMT, every instruction writes the same register number with a different value in each lane. When ADD R0, R0, %threadIdx commits, lane 0 must hold R0 = 0 while lane 3 holds R0 = 3; with one shared R0, the last writer wins and all four threads become thread 3 — vector add fills c with four copies of a[3]+b[3]. "Same register name" in SIMT means same role, never same storage: 4 \times 16 = 64 physical registers, no negotiation. Scaled up, this is why real SMs carry those outlandish 256 KB register files — thousands of threads, each owning its private copy of "R0".