Heat Walls and the End of Scaling

Everything in this course could, until now, be read as an elegant curiosity — reversible gates and Landauer's floor as intellectual furniture. This lesson is where the curiosity turns into a deadline. The reason the whole industry is suddenly interested in the thermodynamics of computing is a chain of walls it has run into: Dennard scaling broke, the power wall froze clock speeds, Koomey's law — the steady fall in energy per operation — is flattening, and its extrapolation runs straight into the Landauer floor within a few decades. When that happens, reversibility stops being optional.

Two scaling laws, one of them dead

Moore's law — transistor counts doubling — is the famous one, and it limped on for years. Its quieter partner, Dennard scaling, was the one that actually mattered for heat. Dennard's 1974 observation was that as you shrink a transistor's dimensions by a factor \kappa, you also drop its voltage by \kappa, and the switching energy \tfrac{1}{2}CV^2 falls just fast enough that power per unit area stays constant. You could pack in more transistors and run them faster with no rise in heat density. That is the free lunch that powered forty years of "just wait for next year's chip".

Around 2005 it ended. Voltage stopped scaling: below roughly 1\ \mathrm{V} the transistor's threshold and exploding leakage current set a floor, so V froze while transistors kept shrinking. With V pinned, \tfrac{1}{2}CV^2 no longer fell in step with the transistor count — and power density began to climb.

The power wall

There is a hard practical ceiling on how much heat you can pull off a chip: roughly 100\ \mathrm{W/cm^2} with air or ordinary liquid cooling — comparable to a kitchen hotplate, from a sliver of silicon the size of a fingernail. Push past it and the junctions cook. Once Dennard scaling died, cranking the clock frequency higher meant crashing straight into that wall. So the industry did the only thing it could:

clock speeds froze in the 3–5\ \mathrm{GHz} band, where they have sat for nearly two decades;
multicore arrived — spend the extra transistors on more cores instead of a faster one, since you cannot afford to run one core harder;
dark silicon emerged — a growing fraction of the transistors on a chip must sit powered off at any instant, because lighting them all at once would exceed the power budget. You can build them; you just cannot afford to switch them all.

Every one of these is a symptom of heat, not of running out of transistors. The machine was no longer limited by how small we could make things, but by how much waste heat we could carry away.

Koomey's law, and where it is heading

Behind Moore and Dennard runs a third trend, Koomey's law: the number of computations you get per joule has, historically, doubled about every 1.6 years — energy per operation halving on the same clock. That is the trend that actually matters for this course, because it is a line marching downward toward a floor that does not move. Plot it on a log axis and extrapolate:

Today's general-purpose operations sit some 10^{5}–10^{6} times above kT \ln 2. At the historical Koomey rate — about one order of magnitude per decade — that is only five or six decades of headroom. Worse, the rate itself has been flattening since around 2000 as the easy voltage and capacitance savings ran out. Whether the crossing lands in the 2040s or slips to the 2060s, the qualitative fact is fixed: conventional irreversible computing is on a collision course with a wall that no manufacturing advance can move, because it is set by k, T and \ln 2 alone.

Why the economics eventually force the issue

This is no longer only a chip-designer's problem. Data centres already consume on the order of 1–2\% of world electricity, and the arrival of large-scale AI is pushing that share up sharply. When a growing slice of civilisation's energy goes into computing, the joules per operation stops being a footnote and becomes a line in the national accounts. And here is the punchline of the whole module: once conventional efficiency gains stall against the Landauer floor, the only remaining source of large improvement is to stop paying the floor at all — to compute reversibly, erasing as few bits as possible and recycling the energy of the rest. Reversibility, for sixty years an elegant theoretical option, becomes an economic necessity.

Dennard scaling (constant power density as transistors shrink) broke around 2005 when supply voltage stopped scaling — pinned by threshold and leakage;
the power wall (~100 W/cm² cooling ceiling) then froze clock speeds at a few GHz and forced multicore and dark silicon — all symptoms of heat, not of transistor count;
Koomey's law (energy/op halving ~every 1.6 yr) is flattening; extrapolated, it meets the fixed Landauer floor within a few decades;
with computing a growing share of world energy, reversibility shifts from optional to mandatory — the only escape once the floor is reached.

Because the industry stopped selling frequency and started selling parallelism and specialisation. When a single core could no longer be pushed past a few gigahertz without melting, chip designers spent their transistor budget differently: more cores, wider vector units, and above all specialised hardware — GPUs for graphics and neural nets, tensor units, video codecs, cryptography blocks. A dedicated circuit does its one job with far fewer wasted switches than a general-purpose core faking it in software, so performance-per-watt keeps improving even with the clock frozen. That is also why "dark silicon" is not pure loss: much of the chip that sits dark at any moment is specialised blocks waiting their turn, each far more efficient than the general core when it is that block's turn to light up. The free lunch of frequency is over; the à-la-carte menu of accelerators is how we still get faster.

Two different walls are easy to blur. The power wall (~100 W/cm²) is an engineering limit — how fast we can carry heat away with fans and cold plates — and it sits some 10^{5} times above the Landauer floor. It is the wall we are actually pressed against today, and it is soft: better cooling, lower voltage, and specialised circuits keep nudging it. The Landauer floor (kT\ln 2) is a fundamental limit that no cooling or cleverness can move, and we are still far below it — for now. The mistake is to say "chips are hot because of Landauer": they are hot because of \tfrac{1}{2}CV^2 engineering waste, roughly a hundred-thousand times the fundamental cost. The story of this lesson is that the two walls are on a slow collision course — the engineering trend line descending toward the fundamental floor — not that we have arrived.