The Streaming Multiprocessor

Look at a die photo of a modern GPU and squint. Underneath the marketing renders is something almost agricultural: the same rectangle, stamped out in neat rows, over a hundred times. That rectangle is the Streaming Multiprocessor — the SM — and it is the GPU's entire personality in one repeatable tile. In the last module you measured the machine from the outside: FLOP/s, bandwidth, the roofline. This module opens the lid. Everything a GPU does — every warp scheduled, every register read, every divergent branch and coalesced load — happens inside an SM, and the whole chip is just that story times a hundred, plus a big L2 cache and some memory controllers to feed it.

Learn one SM thoroughly and you have learned the GPU. That is this lesson's tour.

The tour: four partitions and a shared ground floor

A representative SM is organised as four processing partitions — think of them as four quarter-SMs living in one building. Each partition owns, privately:

Downstairs, shared by all four partitions, sit the L1 data cache / shared memory block (one pool of SRAM, split between the two roles) and the instruction cache. Step through the floor plan:

The numbers, in one table

Vendors vary the exact figures generation to generation, but a representative modern SM looks like this — worth absorbing, because every performance argument in this module is arithmetic over these quantities:

ResourcePer partitionPer SM (×4)
Warp schedulers14
FP32 / INT32 lanes ("CUDA cores")32128
Register file64 KB256 KB
Tensor cores14
Special function units (SFUs)416
Load/store units832
Resident warp slots1248 (some designs 64)
L1 / shared memory128 KB (shared)

Read the warp-slot row twice. An SM doesn't just run warps — it houses them: up to 48 or 64 warps' complete state (program counters, registers, predicate masks) is held on-chip simultaneously, all the time. That is 1,536–2,048 thread contexts resident in one tile. The "multitude of threads" you met when you first saw SIMT is not a software abstraction — it is physically installed in the register file, which is exactly what makes switching between warps free. The next two lessons are entirely about that fact.

From tile to chip: why the GPU is a repeated rectangle

Zoom back out. A full GPU is dozens to hundreds of identical SMs, wired to a large L2 cache (megabytes, shared by everyone) and a ring of memory controllers talking to GDDR or HBM DRAM. That's the whole chip: tile × N, plus plumbing. The tiling is not laziness — it is the business model:

This is why GPU generations can double their SM count and just… work. Nothing inside the SM needed to change; the tile was the unit of both design and repair.

Until 2006, graphics chips had different hardware for vertex work and pixel work — and one kind always sat idle while the other was the bottleneck. NVIDIA's G80 (the GeForce 8800) made the bet that defines the modern GPU: replace both with one pool of identical, general-purpose multiprocessors and schedule any work onto any tile. The unified tile needed a name, and "Streaming Multiprocessor" stuck. The same bet made CUDA possible a year later: if the tile is general enough to run vertex shaders or pixel shaders, it is general enough to run your physics simulation — and, eventually, your neural network. Every SM you'll study in this module is a descendant of that one decision.

Spec sheets say things like "16,384 CUDA cores", and the CPU-trained brain hears "16,384 times a CPU core". It isn't. A CUDA core is an ALU lane — one 32-bit arithmetic pipeline. It has no fetch, no decode, no scheduler, no program counter of its own; thirty-two of them execute one warp's single instruction together, steered by the partition's one scheduler. The honest CPU analogy: a CPU core with two 8-wide AVX units also has "16 lanes", yet nobody sells it as a 16-core chip. Comparing GPU "cores" to CPU cores by count is marketing arithmetic — the comparable unit is the SM (or partition) versus the CPU core, and there are ~100 of those, not 16,000. Impressive enough without the inflation.