GPU Architecture and SIMT

A CPU core is a sprinter obsessed with latency: it will spend a billion transistors on out-of-order execution, branch prediction and giant caches so that one thread finishes as fast as physically possible. A GPU makes the opposite bet. It doesn't care how long any single thread takes; it cares about throughput — total work per second across thousands of threads. It spends its transistors on arithmetic units, not on making one thread clever, and it hides every stall not by predicting around it but by having so many threads ready that there is always something else to run.

This is why a GPU can have tens of thousands of threads in flight and a CPU has a few dozen. It's a different animal for a different job: the CPU wants your email to feel instant; the GPU wants to multiply two enormous matrices, shade a million pixels, or train a network — jobs made of the same operation repeated on mountains of independent data.

SIMT: threads that secretly march in lock-step

NVIDIA calls the GPU's execution model SIMTSingle Instruction, Multiple Threads. To the programmer you write one ordinary-looking function (a "kernel") describing what one thread does, and launch it across a grid of thousands of threads. But underneath, the hardware bundles threads into groups of 32 (NVIDIA calls a group a warp; AMD uses 32- or 64-wide wavefronts) and runs every thread in a warp in lock-step, executing the same instruction on its own data at the same time.

So SIMT is really SIMD wearing a friendlier mask. The lanes of a packed-SIMD unit are exposed to you as independent "threads," which is far nicer to program — each thread has its own registers and its own index — but the machine underneath still issues one instruction per warp per cycle and applies it across all 32 lanes. You get SIMD efficiency with a thread's-eye programming model.

Inside the machine: SMs, schedulers, and a memory hierarchy

A GPU is a grid of Streaming Multiprocessors (SMs; AMD calls them Compute Units). Each SM holds many warps resident at once — their registers all live in a huge register file — and a warp scheduler that, every cycle, picks a warp that is ready and issues its next instruction. The memory hierarchy is tuned for bandwidth: a small fast shared memory (scratchpad) and L1 per SM, an L2 shared across the chip, and high-bandwidth DRAM (GDDR or HBM) feeding the whole thing.

The killer trick: hide latency with threads, not caches

A DRAM access costs hundreds of cycles. A CPU fights that with big caches and out-of-order execution to find independent work. A GPU does something audacious: when a warp issues a memory load and stalls, the scheduler simply switches to another ready warp — instantly, because all warps' state is already sitting in the register file. With enough resident warps, there is always one ready to run, and the memory latency is completely hidden behind other warps' arithmetic. This is why GPUs need so many threads: not for the parallel result, but to have enough spare work to cover the stalls.

The fraction of the possible warp slots you actually keep resident is called occupancy. Too few warps and a memory stall shows through as idle arithmetic units; enough warps and the latency vanishes. The model below shows the tipping point — how many warps you need before the units stay busy:

// Latency hiding: how many warps must be resident to keep the ALUs busy through a memory stall? // If a load costs `latency` cycles and each warp issues an instruction that takes `issue` cycles, // you need about latency/issue warps to always have one ready. const memLatency = 400; // cycles to reach DRAM const issueCycles = 4; // cycles between a warp's successive issues while running const warpsNeeded = Math.ceil(memLatency / issueCycles); console.log(`DRAM latency: ${memLatency} cycles`); console.log(`warps needed to fully hide it: ~${warpsNeeded}`); for (const resident of [8, 32, 64, 128]) { const hidden = Math.min(1, resident / warpsNeeded); const utilisation = (hidden * 100).toFixed(0); console.log(`${resident} resident warps -> ~${utilisation}% ALU utilisation`); } console.log("More threads don't finish faster individually — they keep the machine FULL.");

Warp divergence: the cost of an "if"

Because a warp shares one instruction stream, branches are its Achilles' heel. If some threads in a warp take the \texttt{if} and others the \texttt{else}, the hardware can't run both at once. It runs the then-path with the else-threads masked off, then the else-path with the then-threads masked off — serializing the two sides. This is warp divergence, and in the worst case (every thread down a different path) a 32-wide warp can lose up to 32\times of its throughput. Data-dependent branching, so cheap on a CPU with a good branch predictor, is the single thing that most hurts GPU code. The fix is to arrange data so threads in the same warp usually agree on branches.

CPU vs GPU, side by side

CPU (latency-oriented)GPU (throughput-oriented)
Goalfinish one thread ASAPmaximise total work/second
Cores / threadsa few big cores, dozens of threadsthousands of tiny threads (warps)
Transistor budget spent onOoO, branch prediction, big cachesarithmetic units, huge register file
Hides memory latency bycaches + out-of-order executionswitching to another ready warp
Hatescache misseswarp divergence (branchy code)

GPUs were built to shade pixels: apply the same lighting maths to millions of independent fragments — embarrassingly parallel, arithmetic-heavy, latency-tolerant. It turned out that a neural network's core operation, matrix multiplication, has exactly that shape: the same multiply-accumulate over vast regular arrays with no branching. When NVIDIA exposed the hardware for general computing with CUDA (2007), researchers discovered a graphics card was an astonishingly cheap matrix engine. The modern AI boom rides on a machine designed for video games — and today's data-center GPUs add tensor cores, units that do a whole small matrix-multiply per instruction, doubling down on the coincidence.

Two traps. First, a GPU "thread" is really a SIMD lane dressed up — thousands of them share instruction streams in warps; it is nothing like a heavyweight OS thread with its own program counter and stack switched by the kernel. Second, launching more threads does not make any single thread finish sooner — a GPU is slow per-thread on purpose. More threads raise throughput and occupancy (so stalls stay hidden), not latency. If you need one answer fast and there's no parallelism to exploit, a CPU will beat a GPU every time.