From Toy to Titan

PrimerGPU works — measured, self-checked, cycle-accounted. So what stands between the machine you built and the 80-billion-transistor monster rendering games and training models? This closing audit answers that the way a working engineer would: not with awe, but with a gap list — what did we deliberately leave out, and in what order would you add it back? The punchline is worth stating up front, because it is this course's whole thesis about GPUs: the gulf between toy and titan is roughly a thousandfold in numbers and roughly zero in concepts. Every organ of a real GPU is an organ your machine already has, grown enormous — plus a handful of new ones you have already met in earlier modules. What you do not get for free is the engineering. We will be honest about both halves.

The growth plan: five stages

Suppose you were told to evolve PrimerGPU into a real product. Here is the order that respects the dependencies — correctness first, memory second, throughput machinery after, specialisation last:

Stage 1 — general control flow and real multithreading. First, replace the lane-0 branching shortcut with genuine divergence machinery — active masks, both-paths execution, reconvergence — because until then, whole classes of kernels are silently wrong. Then widen: 32 lanes per warp, many warps resident per core, and a real warp scheduler with a scoreboard, so those WAIT cycles our budget table exposed get filled with other warps' work instead of silence. This one stage is most of the distance from our core to a true streaming multiprocessor;
Stage 2 — a memory hierarchy. Programmer-managed shared memory with its banked structure, L1 and L2 caches, and a grown-up coalescer handling misalignment, strides and partial merges. Our two naked channels made latency visible; a hierarchy is how real machines keep the average request from ever reaching DRAM;
Stage 3 — occupancy machinery. Let each core host several blocks at once, which turns the register file into a budget: how many warps fit depends on how many registers each thread claims. This is bookkeeping hardware — allocators, limit checks — that exists purely to keep stage 1's scheduler fed;
Stage 4 — the graphics organs. Texture units with their filtering arithmetic, the rasterizer, render output units — fixed-function machinery from the graphics-hardware module that shares the SMs with compute. This is where a compute engine becomes a graphics processing unit again;
Stage 5 — the modern frontier. Tensor engines for matrix math, HBM stacks on interposers, and scaling past the reticle with chiplet construction — the gpu-frontiers module, bolted onto your floor plan.

Notice the logic of the ordering: stages 1–2 are about being correct and tolerable on general code, stage 3 amplifies 1, and stages 4–5 are profitable specialisations that only make sense once the general machine underneath is sound.

Two die plans, side by side

This is the picture to keep: the titan is not a different kind of machine. It is your block diagram under a magnifying glass — with the five growth stages filling in the details.

The numbers: ~1000× in size, ~0× in concepts

Quantity	PrimerGPU	A modern SM / GPU	Gulf
Lanes per core	4	128 FP32 lanes per SM	32×
Threads resident per core	4 (one block)	up to 2,048 (64 warps)	512×
Register file per core	128 bytes	256 KB per SM	2,048×
Shared memory / L1	none	~228 KB per SM	∞
Cores on the chip	2	100–150 SMs	~60×
Threads on the chip	8	~270,000 resident	~34,000×
Memory system	2 channels, 1 word / 4 cycles	HBM stacks, terabytes per second	~10⁶×
Concepts	SIMT lockstep · index registers · block dispatch · arbitration · coalescing — identical		1×

And the bottom row is not a joke — it is the audit's finding. What the toy got right is precisely the load-bearing skeleton: one decoder amortised over lockstep lanes; thread identity as read-only registers; blocks as the unit of scalable dispatch; scarce memory ports behind arbitration, made tolerable by coalescing; and a cycle budget dominated by WAIT, which is the very problem the titan's extra thousands of threads exist to bury. Someone who understands the left column has understood the right column's reasons. What remains is scale — and scale, as the next card insists, is not nothing.

growth order: divergence + warps + scheduler → memory hierarchy → occupancy → graphics fixed-function → tensor/HBM/chiplets;
the gulf is ~10²–10⁶ in every quantity, ~1× in concepts;
the toy's honest skeleton: SIMT lockstep, index registers, block dispatch, memory arbitration + coalescing;
the toy's honest disease — WAIT domination — is the titan's design driver: massive residency exists to hide it.

Surprisingly little, structurally — which is the best evidence that the concepts truly carry. Today one core holds one PC, one instruction register, one FSM position and four lanes of registers. Warp support means turning that singular state into an array: per-warp PCs and FSM positions, a register file partitioned per warp, and — the only genuinely new circuit — a little scheduler that each cycle picks a warp whose state says "ready" and lets it use the shared fetch/decode/lanes for that step. Our WAIT state already marks exactly the moments a warp is not ready; the scheduler simply issues someone else then, and the idle cycles from our budget table get eaten one by one. That is a weekend of simulator hacking and a very instructive month of RTL — and it is precisely the machine you studied in the GPU-compute module, rediscovered as the natural next commit to your own codebase.

The audit says the ideas transfer; do not let that curdle into "so the rest is mere typing". The rest is a decade of team-years, and it goes where this course's other modules pointed: verification at a scale where the state space laughs at your testbench and a missed corner case ships in a hundred million devices; physical design, where clock distribution, power delivery and timing closure across an 800 mm² die are research problems, not chores; and manufacturing, where yield economics decide which architectures are even allowed to exist. The five growth stages are each "a lesson's worth of concept" and "a hundred engineer-years of execution" — and the ratio between those two numbers is exactly why the industry needs people who have walked the whole flow, RTL to GDSII, the way you now have. Conceptual mastery is the entry ticket, not the destination.