Emerging Architectures

For fifty years the answer to "how do we make computers faster?" was the same: wait. Transistors shrank on schedule (Moore's law), and shrinking them also made them faster and cooler per operation (Dennard scaling), so every generation you got more, faster transistors almost for free. That era is over. Dennard scaling broke around 2005 — hence the power wall — and Moore's law has slowed to a crawl as we approach atomic dimensions. So the field has pivoted from "wait for the transistors" to "be clever about the ones we have".

Hennessy and Patterson called this a "new golden age" for computer architecture, and they meant it as good news: with the free lunch gone, the interesting gains now come from architecture — packaging, specialisation, moving computation to the data, and opening the instruction set. This final lesson is a tour of where the machine goes next.

Chiplets: stop building one big die

The first big shift is in how you physically assemble a chip. Historically a processor was one monolithic slab of silicon. But manufacturing defects are random and roughly uniform per unit area, so the chance a die is defect-free falls exponentially with its area — a giant die is far more likely to contain a killer defect, and yield craters. Worse, the biggest die a lithography machine can print in one shot (the reticle limit, ~800 mm²) is a hard ceiling.

The fix — pioneered at scale by AMD's Zen processors — is chiplets: build the processor from several small dies and connect them in one package. Small dies yield much better, you can mix process nodes (expensive cutting-edge silicon for the cores, cheap mature silicon for I/O), and you dodge the reticle limit. The code below shows the yield economics that make this a landslide, using the standard exponential defect model.

// Die-yield economics: why 4 small chiplets beat 1 big die. // Poisson defect model: yield = exp(-defectDensity × area). Cost per GOOD die ∝ area / yield. const D = 0.10; // defects per mm^2 (illustrative) const bigArea = 600; // one monolithic die, mm^2 function yieldOf(area: number): number { return Math.exp(-D * area); } function costPerGood(area: number): number { return area / yieldOf(area); } // proportional cost // Monolithic: one 600 mm^2 die. const mono = costPerGood(bigArea); // Chiplet: four 150 mm^2 dies (same total silicon) + a packaging surcharge. const chipletArea = bigArea / 4; const packaging = 1.15; // 15% packaging/interposer overhead const chiplet = 4 * costPerGood(chipletArea) * packaging; console.log(`monolithic yield (600mm^2): ${(yieldOf(bigArea) * 100).toFixed(1)}%`); console.log(`chiplet yield (150mm^2): ${(yieldOf(chipletArea) * 100).toFixed(1)}% each`); console.log(`relative cost/good monolithic: ${mono.toFixed(0)}`); console.log(`relative cost/good 4 chiplets: ${chiplet.toFixed(0)}`); console.log(`chiplets are ${(mono / chiplet).toFixed(1)}x cheaper per working part`);

with roughly uniform random defects, yield falls about exponentially with die area: Y \approx e^{-D \cdot A} (defect density D, area A);
cost per working die scales like A / Y, so splitting one big die into several small ones sharply lowers cost — plus it beats the reticle-size limit;
the price is packaging complexity and die-to-die interconnect, which advanced packaging exists to solve.

2.5D and 3D: building upward

Chiplets need somewhere to talk. Advanced packaging supplies it. 2.5D integration mounts several dies side-by-side on a silicon interposer — a carrier etched with thousands of ultra-dense wires — so chiplets communicate almost as if they were one die. 3D integration goes vertical, stacking dies atop one another and connecting them with through-silicon vias (TSVs) punched straight through the wafer. High-Bandwidth Memory (HBM) is the poster child: a tower of DRAM dies stacked and TSV-connected, parked millimetres from the compute for enormous bandwidth. Apple's M-series "UltraFusion" fuses two dies this way; NVIDIA's AI accelerators ring their GPU with HBM stacks on a TSMC CoWoS interposer.

Approach	Idea	Win	Catch
Monolithic	one big die	simple, fast on-die wires	poor yield, reticle limit
Chiplets (2.5D)	small dies on an interposer	yield, mixed nodes, scale	interposer cost, die-to-die latency
3D stacking	dies stacked with TSVs	huge bandwidth, tiny footprint	heat is trapped inside the stack

Move the compute to the data: near- and in-memory computing

The memory wall says that fetching data across the bus costs far more time and energy than the arithmetic you do on it — often 100× more energy to move a word than to add it. So a radical idea gains ground: stop shipping data to the processor and instead do the computation inside (or beside) the memory. Processing-in-memory (PIM) puts simple ALUs on the DRAM dies themselves (Samsung's HBM-PIM, UPMEM's DIMMs), turning memory bandwidth that used to be stranded into usable compute for data-hungry workloads like search, analytics and AI inference. It inverts the century-old assumption that computing happens in the CPU and memory just stores.

Open silicon and specialised silicon

Two more currents are reshaping the field. First, RISC-V: an instruction-set that is open and royalty-free, so anyone — a startup, a university, a hyperscaler — can build a compliant chip without licensing x86 or ARM. Its clean, modular, extensible design has made it the default for research and a fast-growing force in industry, from tiny microcontrollers to the control cores buried inside GPUs. An open ISA does for hardware what open source did for software.

Second, with general-purpose scaling stalled, the gains now come from domain-specific architectures — chips shaped to one job. Google's TPU for tensor maths, video transcoders, crypto engines, and network processors all trade generality for spectacular efficiency on their niche. Two experimental frontiers push further: neuromorphic chips (Intel's Loihi, IBM's TrueNorth) that mimic spiking neurons for ultra-low-power sensing, and — a different kind of machine entirely — quantum computers, which we treat in their own course rather than here, because they compute on a fundamentally different physical principle.

Where architecture goes next

The through-line of this whole course has been a single question — how do you get more useful work per second, per watt, per dollar? For decades the transistor answered it. Now the architect must, and the answers are more varied and more interesting than ever: assemble chips from chiplets, stack them in three dimensions, compute inside the memory, open the instruction set, and specialise ruthlessly for the workloads that matter. The free lunch is gone — but that is exactly what makes this a golden age to be designing machines.

In 2017 AMD was the underdog, and its Zen architecture made a contrarian gamble: build server processors from several small identical chiplets rather than one huge die. The yield maths (you just ran a version of it) was decisive — small dies are dramatically cheaper per working part, and AMD could reuse one chiplet design across desktop and server, scaling core counts by simply adding more. Intel, with the world's best monolithic manufacturing, stayed the course longer and got badly stung when its next process node slipped: a monolithic strategy has no fallback when one big die won't yield. Within a few years chiplets had gone from heresy to the industry's default, and Intel too moved to a "tile" (chiplet) design. A packaging decision reshuffled the entire competitive landscape.

Two seductive misconceptions. First, stacking dies vertically multiplies performance density — but it also traps heat: the inner dies of a 3D stack cannot easily shed their power, so thermals, not transistors, become the limit. This is why 3D is used first for memory (low power per die) and only cautiously for hot logic. Second, "RISC-V is open" describes the instruction set, not the chip: an open ISA lets anyone design a core, but a good RISC-V chip still needs a superb microarchitecture, fab access, and years of engineering. Openness removes a licensing tollbooth; it does not hand you Apple-class silicon for nothing. An ISA is a contract, not a chip — as this course said on day one.