Strip away the marketing and a modern GPU is a bandwidth machine. Tens of thousands of lanes, hundreds of matrix engines, ray-tracing units — all of it exists to convert a torrent of bytes into a torrent of results, and all of it stalls the instant the bytes stop coming. A datacentre GPU sustains multiple terabytes per second from its DRAM; a CPU in the same rack makes do with a tenth of that. Feeding this appetite is not a detail delegated to "the memory vendor" — it has bent the DRAM industry into building two entirely different kinds of chip, sold to two entirely different markets, soldered centimetres — or micrometres — from the GPU die.

This lesson is about those two answers to the same question. GDDR makes commodity DRAM fast: heroic signalling over an ordinary circuit board. HBM makes DRAM close: stacked dice wired straight down into the package your packaging lesson built. Same silicon physics, opposite engineering bets — and the arithmetic that decides between them is short enough to do in your head.

Bandwidth is pins times speed

Every memory interface obeys one embarrassingly simple equation:

A "384-bit" GPU has 384 data wires to its DRAM; if each wire carries 32 Gb/s, the board moves 384 \times 32 / 8 = 1536 GB/s — about 1.5 TB/s. That's the whole product. Everything interesting about GDDR versus HBM is a fight over which factor to grow: make each pin ferociously fast, or make the bus absurdly wide. The two choices sound symmetric. They are not — because energy, cost and board area punish the two factors very differently.

GDDR: the wide-and-fast commodity path

GDDR is what you get when you keep DRAM as a separate, cheap, mass-produced chip soldered to the board, and pour all the heroism into the wires. Each GDDR device talks to the GPU over a short, carefully tuned point-to-point link — no shared bus, no DIMM connectors, every trace length-matched on the board — and the per-pin rates are extreme: GDDR6 runs 16 Gb/s/pin, GDDR6X pushed past 21, and GDDR7 reaches 32 Gb/s and beyond. A flagship gaming board with a 384-bit bus (twelve devices, 32 bits each) lands at roughly 1–1.5 TB/s.

How does a copper trace on a consumer PCB carry 32 Gb/s? Partly by ceasing to be binary: GDDR6X and GDDR7 use PAM signalling (pulse-amplitude modulation) — the wire carries multiple voltage levels per symbol instead of two, so each clock edge delivers more than one bit. More bits per edge, at the price of shrunken voltage margins and exquisite sensitivity to noise.

And there is the bill. Driving a signal that fast off-chip, across a board costs on the order of 7 pJ per bit. Each device needs its own tuned point-to-point routing, so the board fills with length-matched serpentine traces; the GPU die spends a thick ring of edge area on high-speed PHYs; and per-device capacity is modest (a few GB each), capping a 384-bit board at a few tens of GB. GDDR's virtue is that the DRAM itself stays a commodity: standard packaging, standard assembly, cheap per gigabyte. The heroism lives in the signalling, and you pay for it in picojoules and board complexity.

HBM: the stacked-and-close path

HBM starts from the opposite premise: stop shouting across the board — move in together. Take four, eight, twelve DRAM dice and stack them vertically, wired through each other by through-silicon vias (TSVs) — the drilled-and-filled vertical wires from the packaging lesson — down to a logic base die. Then place the whole stack millimetres from the GPU on a 2.5D silicon interposer, whose lithographically-drawn wiring is fine enough to route a bus no circuit board could ever carry: 1024 bits wide per stack (2048 in HBM4), thousands of connections in the space of a fingernail.

With that much width, nobody needs heroic pins. Each HBM wire runs at a modest 6–10 Gb/s — a rate so relaxed it needs no PAM, no board tuning, no equalisation heroics. Do the arithmetic:

— one stack ≈ 1 TB/s, and a datacentre GPU carries six or eight stacks: 6–8 TB/s aggregate. And because each bit travels micrometres of interposer instead of centimetres of FR4, the energy is roughly halved: ~3.5 pJ/bit against GDDR's ~7. Capacity stacks too — vertically: 24–36 GB per stack, 150–300 GB per GPU.

Feel the wattage. A 1.5 TB/s interface streaming flat out moves 1.5 \times 10^{12} \times 8 = 1.2 \times 10^{13} bits every second. At 7 pJ/bit that is 84 W — just for the memory link, before a single flop — while the same bandwidth at 3.5 pJ/bit costs 42 W. Scale to a 6 TB/s HBM machine and the interface alone is a three-digit wattage either way; at GDDR energies it would be untenable. This is why "energy per bit" — not peak bandwidth — is the number memory architects tattoo on their forearms.

The trade, in one table

Read the last row with the others in view and the market split explains itself. A gaming GPU sells into a price-fanatical market, needs "only" ~1 TB/s, and lives on a board with room for serpentine traces: GDDR, obviously. A training GPU sells for tens of thousands of dollars, is bought by the megawatt, and is throttled by memory bandwidth before anything else: HBM, obviously. Neither is "better memory". They are different answers to different constraints.

One more cost is easy to miss because it's on the GPU, not the DRAM: the controller and PHY silicon. Every memory channel needs a scheduler — the request-reordering, bank-juggling machinery from memory controllers — plus a PHY driving the pins. On a big GPU the ring of memory controllers and PHYs eats a substantial slice of the die edge, whichever technology you pick: GDDR spends it on blistering per-pin transceivers, HBM on sheer beachfront width facing the interposer. Memory is not a peripheral you bolt on; it is a co-designed organ — die floorplan, package, and DRAM engineered as one system.

Two bar charts, two winners

Here are the same two memories scored twice — once per watt, once per dollar. Step through and watch the ranking flip.

Run the arithmetic yourself

The whole comparison fits in twenty lines. Pins × rate ÷ 8 gives bandwidth; bytes × 8 × pJ/bit gives energy. Change the numbers — try a 512-bit GDDR board, or HBM4's 2048-bit stacks — and watch the trade move.