Photonics in the Datacenter

Take a walking tour of a modern warehouse-scale computer and count the copper. Inside a server: copper. Server to top-of-rack switch, a metre or two: copper twinax, while it still can be. And then — nothing but glass. Rack to aggregation, aggregation to spine, spine to the campus ring, the campus to the submarine cable: every link beyond a few metres has been optical for years. The quiet punchline of this whole course is that photonic computing's greatest victory is already deployed at planetary scale — not as an exotic processor, but as the circulatory system of every datacenter on Earth. This closing lesson tours the frontier: where the copper–glass boundary sits today, the moment optics stopped merely carrying for the network and started being the network, and the machine-learning bandwidth explosion now dragging photons toward the GPU itself.

The boundary, and its direction of travel

The crossover chart from lesson one is drawn on the datacenter floor in cable colours. At 25 Gb/s per lane, a passive copper direct-attach cable (DAC) reached 5 m — plenty for a rack. At 100 G/lane it reaches barely 2 m; at 200 G/lane, about one. The bandwidth-distance product is doing exactly what lesson one promised: each lane-rate doubling halves copper's reach, so the optical boundary marches inward one network tier per generation — spine (long ago), aggregation, top-of-rack, and next the rack's own interior. Meanwhile the cost per optical bit keeps falling as transceivers ride silicon-photonics economics, and co-packaging waits at the end of the road. Nothing in physics pushes back the other way: the boundary only ever moves toward the die.

Apollo: mirrors where the spine used to be

For a decade the datacenter's optics stopped at the switch faceplate: light carried the bits, but every hop converted them back to electrons for packet switching. Google's Apollo deployment broke that pattern at the boldest possible place — the spine, the top tier of the Jupiter fabric through which whole aggregation blocks talk to each other. In Apollo, the spine's electrical packet switches are gone, replaced by Palomar optical circuit switches: 136×136-port 3-D MEMS mirror units — the millisecond, rate-agnostic, low-loss technology from the previous lesson — that simply steer each block's fibres directly at another block's.

Why does circuit switching — with its telephone-exchange manners — work at the heart of a packet network? Aggregation. Individual flows are bursty, but the traffic between two thousand-server blocks is a statistical average that drifts on minutes-to-hours timescales, so a patch panel that re-points in tens of milliseconds is effectively instantaneous. The wins compound: a mirror consumes essentially no power and needs no transceivers (the spine tier previously burned two per link plus a switch ASIC's share); and because mirrors don't care what the light carries, the same OCS survived three generations of link-rate upgrades — 40G to 100G to 200G and beyond — without being touched. Google reports the approach cut fabric cost by roughly 30% and fabric power by around 40%, while making topology itself a schedulable resource.

The ML bandwidth explosion

Then machine learning detonated the demand curve. Scaling laws reward ever-larger models trained on ever-larger clusters — and a training step is a synchronised operation: every accelerator must exchange gradients with its peers before anyone proceeds. All-reduce arithmetic makes the appetite concrete: a 100-billion-parameter model has ~200 GB of 16-bit gradients per step; a ring all-reduce moves roughly twice that through every chip's links every step, several steps per second. Multiply out and the cluster's internal traffic dwarfs its traffic to the outside world — which is why an accelerator now ships with more link bandwidth than some datacenters had spines:

// All-reduce appetite of one training step (ring algorithm: ~2× data through each chip). const paramsB = 100e9; // parameters const bytesPerParam = 2; // bf16 gradients const gradientsGB = paramsB * bytesPerParam / 1e9; const perChipGBs = 900; // scale-up link bandwidth per accelerator, GB/s const stepCommSeconds = 2 * gradientsGB / perChipGBs; console.log("gradients: " + gradientsGB + " GB; comm per step: " + stepCommSeconds.toFixed(2) + " s"); console.log("at 2 steps/s this must overlap compute -> links must stay saturated"); // The photonic rule of thumb from the ONoC lesson: full pipes favour photons. const clusterChips = 4096; const totalTBs = clusterChips * perChipGBs / 1000; console.log("cluster scale-up bandwidth: " + totalTBs + " TB/s, held busy every step");

Full pipes, long distances, thousands of endpoints: the exact conditions last lesson's autopsy said favour photons. And so, at cluster scale, they got them. Google's TPUv4 supercomputer wires 4,096 chips into a 3-D torus whose inter-cube faces pass through optical circuit switches: the OCS layer lets schedulers carve out arbitrarily-shaped slices, re-twist the torus for a job's communication pattern, and route around failed racks by re-pointing mirrors — topology as software. GPU fabrics are converging on the same physics from the other side: a scale-up domain like NVLink pushes terabytes per second per device, copper caps it at a couple of metres — one rack — and breaking past that wall means optical I/O on the GPU package: co-packaged engines, dense WDM, pJ/bit budgets. Every thread of this module, braided into one cable.

Seeing "optical switch" in a datacenter diagram, it is fatally easy to imagine a faster Ethernet switch. An OCS is a different kind of thing: a motorised patch panel. It makes no per-packet decisions — it cannot, since (as the fabrics lesson established) it can neither buffer a photon nor read a header. While its mirrors tilt, light on the moving path is simply lost — tens of milliseconds of darkness, an eternity beside a 100 ns packet time. The deployments succeed because they never ask the OCS to do a packet switch's job: reconfiguration is scheduled (between training jobs, or as traffic matrices drift), packet-level statistical multiplexing still happens in the electrical switches at the edges, and the OCS provides cheap, rate-proof, reconfigurable capacity between them. Electrons decide; photons carry. The division of labour is the moral of this entire module — an OCS deployment is simply that moral, bolted into a rack.

There is something philosophically delicious about the Palomar OCS: it is a core component of the world's most advanced packet networks that contains no bits. No SerDes, no buffers, no forwarding tables — just 136 beams, an array of two-axis MEMS mirrors a few hundred microns wide, and a feedback camera watching dim alignment beacons to keep each beam centred on its target (insertion loss: about 2 dB, a line item any link budget can absorb). Its ancestors were the telecom optical cross-connects of the dot-com era — built for a bandwidth boom that arrived a decade late, many of their makers dead by 2003; the technology waited patiently for a customer whose traffic actually justified it. When the upgrade from 100G to 400G arrived, Google's spine needed no forklift: the mirrors reflected the new rate as indifferently as the old, because a mirror does not know what a baud is. In an industry that measures device generations in months, the most future-proof box in the building turned out to be the one with no electronics in the datapath at all.

Module 8, closed — and the road on

Tally the module's verdict. Photonics lost the logic war (Module 7) but owns transport: it carries every long link (WDM links), is marching into the package (CPO), switches wholesale capacity with mirrors (fabrics, Apollo), and stops — honestly — where distances shrink below the crossover (the ONoC autopsy). Photons carry; electrons compute and buffer. The next module asks the subversive follow-up: what if the photon's quantum nature — the very thing classical links average away — were itself the computational resource? That story begins with photons as qubits; and for the systems view of how all these links are controlled and kept in calibration, Module 11's electronic–photonic interface awaits.