Look at a die photo of a modern GPU and squint. Underneath the marketing renders is something
almost agricultural: the same rectangle, stamped out in neat rows, over a hundred times.
That rectangle is the Streaming Multiprocessor — the SM — and it is the GPU's
entire personality in one repeatable tile. In
Learn one SM thoroughly and you have learned the GPU. That is this lesson's tour.
A representative SM is organised as four processing partitions — think of them as four quarter-SMs living in one building. Each partition owns, privately:
Downstairs, shared by all four partitions, sit the L1 data cache / shared memory block (one pool of SRAM, split between the two roles) and the instruction cache. Step through the floor plan:
Vendors vary the exact figures generation to generation, but a representative modern SM looks like this — worth absorbing, because every performance argument in this module is arithmetic over these quantities:
| Resource | Per partition | Per SM (×4) |
|---|---|---|
| Warp schedulers | 1 | 4 |
| FP32 / INT32 lanes ("CUDA cores") | 32 | 128 |
| Register file | 64 KB | 256 KB |
| Tensor cores | 1 | 4 |
| Special function units (SFUs) | 4 | 16 |
| Load/store units | 8 | 32 |
| Resident warp slots | 12 | 48 (some designs 64) |
| L1 / shared memory | — | 128 KB (shared) |
Read the warp-slot row twice. An SM doesn't just run warps — it houses them: up to 48 or 64 warps' complete state (program counters, registers, predicate masks) is held on-chip simultaneously, all the time. That is 1,536–2,048 thread contexts resident in one tile. The "multitude of threads" you met when you first saw SIMT is not a software abstraction — it is physically installed in the register file, which is exactly what makes switching between warps free. The next two lessons are entirely about that fact.
Zoom back out. A full GPU is dozens to hundreds of identical SMs, wired to a large L2 cache (megabytes, shared by everyone) and a ring of memory controllers talking to GDDR or HBM DRAM. That's the whole chip: tile × N, plus plumbing. The tiling is not laziness — it is the business model:
This is why GPU generations can double their SM count and just… work. Nothing inside the SM needed to change; the tile was the unit of both design and repair.
Until 2006, graphics chips had different hardware for vertex work and pixel work — and one kind always sat idle while the other was the bottleneck. NVIDIA's G80 (the GeForce 8800) made the bet that defines the modern GPU: replace both with one pool of identical, general-purpose multiprocessors and schedule any work onto any tile. The unified tile needed a name, and "Streaming Multiprocessor" stuck. The same bet made CUDA possible a year later: if the tile is general enough to run vertex shaders or pixel shaders, it is general enough to run your physics simulation — and, eventually, your neural network. Every SM you'll study in this module is a descendant of that one decision.
Spec sheets say things like "16,384 CUDA cores", and the CPU-trained brain hears "16,384 times a CPU core". It isn't. A CUDA core is an ALU lane — one 32-bit arithmetic pipeline. It has no fetch, no decode, no scheduler, no program counter of its own; thirty-two of them execute one warp's single instruction together, steered by the partition's one scheduler. The honest CPU analogy: a CPU core with two 8-wide AVX units also has "16 lanes", yet nobody sells it as a 16-core chip. Comparing GPU "cores" to CPU cores by count is marketing arithmetic — the comparable unit is the SM (or partition) versus the CPU core, and there are ~100 of those, not 16,000. Impressive enough without the inflation.