DRAM and Memory Controllers

We have spent the whole module hiding it, but here it finally is: the floor of the hierarchy, the "main memory" that every cache miss eventually falls through to. DRAM — dynamic random-access memory — is a marvel of density and a monument to compromise. It is why you can afford 16 GB of RAM, and it is the wall in "memory wall". Understanding its peculiar internal structure explains one of the most important facts in computer architecture: DRAM latency has barely improved in two decades, even as its bandwidth soared.

One transistor, one capacitor

The entire trick of DRAM is its cell: a single transistor gating a single tiny capacitor. A charged capacitor is a 1, a drained one is a 0. Compare that to SRAM, which needs six transistors per bit to hold its state actively. That 1-vs-6 ratio is the whole story: DRAM packs bits far more densely and cheaply, which is why it's the bulk memory — and SRAM, fast but bulky, is reserved for caches.

	DRAM cell	SRAM cell
Devices per bit	1 transistor + 1 capacitor	6 transistors
Density / cost	very high / cheap	low / expensive
Speed	slow (tens of ns)	fast (≈1 ns)
Needs refresh?	yes — the capacitor leaks	no (as long as powered)
Used for	main memory	caches, registers

That capacitor is DRAM's blessing and curse. It leaks — charge drains away in milliseconds — so every cell must be periodically read and rewritten: refresh. The "dynamic" in DRAM is exactly this need to be constantly topped up. Refresh steals a small slice of bandwidth and power, and it grows more troublesome as chips get denser.

Inside a bank: rows, the row buffer, and columns

A DRAM chip is organised into a 2-D grid of cells arranged in rows and columns, plus a critical piece of SRAM-speed circuitry: the row buffer (a line of sense amplifiers). You cannot read a single DRAM bit directly — the process is two-step:

ACTIVATE (RAS) — select a row; the whole row's charge is sensed and latched into the row buffer. This is the slow, destructive step (it drains the cells, so they're rewritten from the buffer).
READ/WRITE (CAS) — pick columns out of the now-open row buffer. Fast, and repeatable for any column of the open row.
PRECHARGE — before opening a different row, close the current one (write the buffer back, reset the bitlines).

The row buffer is effectively a tiny cache of the last-opened row. If your next access hits the same open row (a row buffer hit), you pay only the fast CAS. If it needs a different row in that bank (a row buffer conflict), you pay PRECHARGE + ACTIVATE + CAS — much slower. Spatial locality at the DRAM level is all about staying within the open row.

Banks, ranks, channels — parallelism to the rescue

A single bank can only do one thing at a time, so DRAM is replicated for parallelism, in a neat hierarchy:

Level	What it is	Buys you
Bank	one cell array + one row buffer	an independent open row; overlap ACTIVATE of one with CAS of another
Rank	a set of chips operated together (banks share the data bus)	more banks to interleave across
Channel	an independent memory bus + controller	true parallel bandwidth (accesses proceed fully in parallel)

This is why DRAM bandwidth has grown enormously (more channels, wider buses, faster signalling) while latency — the time for one lonely access to that leaky capacitor and slow sense amp — has stubbornly stalled. You can pour data through many pipes at once, but the length of any single pipe barely shrinks. Bandwidth is a matter of parallelism you can buy; latency is physics you cannot.

The memory controller: a tiny scheduler

Between the last-level cache and the DRAM sits the memory controller, and it is far cleverer than a passer of requests. It holds a queue of pending accesses and reorders them to exploit exactly the structure above — a policy called FR-FCFS (first-ready, first-come-first-serve): prioritise requests that hit the currently open row (they're "ready", cheap), and interleave across banks and channels to keep them all busy. It also decides the page policy:

Open-page — leave the row open after an access, betting the next access hits it (good with locality). A hit is cheap; a conflict pays the full PRECHARGE penalty.
Closed-page — precharge immediately after each access, betting the next access wants a different row (good for random access). No conflict penalty, but no hit bonus either.

Add refresh scheduling, write draining, and power management, and the memory controller becomes one of the most performance-critical schedulers on the chip. Run the model below to watch open-page policy reward locality — and punish a row-thrashing stream.

// DRAM latency model (cycles). Row-buffer hit = CAS only; conflict = PRECHARGE + ACTIVATE + CAS. const tRP = 14, tRCD = 14, CL = 14; // precharge, activate, column latencies const openHit = CL; // same row already open const conflict = tRP + tRCD + CL; // must close old row, open new one const emptyOpen = tRCD + CL; // bank closed, just open + read // Open-page policy: track the open row per bank; stream of (bank, row). const open: Record<number, number> = {}; function access(bank: number, row: number): number { let cost: number; if (open[bank] === undefined) cost = emptyOpen; else if (open[bank] === row) cost = openHit; // ROW HIT! else cost = conflict; // row conflict open[bank] = row; return cost; } // A locality-friendly burst in row 5, then a jump. const stream: [number, number][] = [[0,5],[0,5],[0,5],[0,5],[0,9],[0,9]]; let total = 0; for (const [b, r] of stream) { const c = access(b, r); total += c; console.log(`bank ${b} row ${r}: ${c} cyc`); } console.log(`total = ${total} cyc (row hits are ${openHit} vs ${conflict} for a conflict)`);

Because DRAM latency is dominated by analog physics, not digital cleverness. Sensing a tiny capacitor's charge through a long, thin bitline is slow, and slower as cells shrink (the capacitor holds less charge, harder to sense). Manufacturers optimise DRAM for the metric customers pay for — density and bandwidth (bits per dollar, bits per second) — not the latency of a single access, which would require larger, costlier cells. So each generation packs more bits and streams them faster, while the raw time to reach one bit for the first time (roughly the CAS + activate) stays near \sim 50\ \text{ns} decade after decade. The memory wall is, at bottom, a capacitor that refuses to be hurried.

The headline number on a memory stick (DDR4-3200, DDR5-6400) is the transfer rate — how many mega-transfers per second it streams, i.e. bandwidth. It says almost nothing about latency, the time to the first byte, which is set by the CAS/activate timings (the "CL16" style numbers) and has hardly changed. A faster DDR generation moves more data per second but a single random access still takes tens of nanoseconds — often the same tens of nanoseconds as the previous generation. This is why doubling your memory's rated speed rarely doubles a latency-bound program: you bought a wider firehose, not a shorter one. Bandwidth and latency are independent axes; know which one your workload is starved for.

The bottom of the hierarchy

You have now walked the whole memory hierarchy top to bottom — from locality and the wall, through caches, translation and the TLB, down to the leaky capacitors of DRAM. Every level exists to hide the level below; every trick is a bet on locality; and at the very bottom is a single transistor guarding a drop of charge, doing its slow analog best to keep a gigahertz processor fed.