Photonics in the Datacenter
Take a walking tour of a modern
warehouse-scale
computer and count the copper. Inside a server: copper. Server to top-of-rack switch,
a metre or two: copper twinax, while it still can be. And then — nothing but glass. Rack to
aggregation, aggregation to spine, spine to the campus ring, the campus to the submarine cable:
every link beyond a few metres has been optical for years. The quiet punchline of this whole
course is that photonic computing's greatest victory is already deployed at planetary
scale — not as an exotic processor, but as the circulatory system of every datacenter on
Earth. This closing lesson tours the frontier: where the copper–glass boundary sits today, the
moment optics stopped merely carrying for the network and started being the
network, and the machine-learning bandwidth explosion now dragging photons toward the GPU itself.
The boundary, and its direction of travel
The crossover chart from
lesson
one is drawn on the datacenter floor in cable colours. At 25 Gb/s per lane, a passive
copper direct-attach cable (DAC) reached 5 m — plenty for a rack. At 100 G/lane it reaches barely
2 m; at 200 G/lane, about one. The bandwidth-distance product is doing exactly what lesson one
promised: each lane-rate doubling halves copper's reach, so the optical boundary marches inward one
network tier per generation — spine (long ago), aggregation, top-of-rack, and next the rack's own
interior. Meanwhile the cost per optical bit keeps falling as
transceivers
ride silicon-photonics economics, and
co-packaging
waits at the end of the road. Nothing in physics pushes back the other way: the boundary only ever
moves toward the die.
Apollo: mirrors where the spine used to be
For a decade the datacenter's optics stopped at the switch faceplate: light carried the bits, but
every hop converted them back to electrons for packet switching. Google's Apollo
deployment broke that pattern at the boldest possible place — the spine, the top tier of the Jupiter
fabric through which whole aggregation blocks talk to each other. In Apollo, the spine's electrical
packet switches are gone, replaced by Palomar optical circuit switches: 136×136-port
3-D MEMS mirror units — the millisecond, rate-agnostic, low-loss technology from the
previous
lesson — that simply steer each block's fibres directly at another block's.
Why does circuit switching — with its telephone-exchange manners — work at the heart of a packet
network? Aggregation. Individual flows are bursty, but the traffic between two
thousand-server blocks is a statistical average that drifts on minutes-to-hours timescales, so a
patch panel that re-points in tens of milliseconds is effectively instantaneous. The wins
compound: a mirror consumes essentially no power and needs no transceivers (the spine tier
previously burned two per link plus a switch ASIC's share); and because mirrors don't care what
the light carries, the same OCS survived three generations of link-rate upgrades — 40G to 100G to
200G and beyond — without being touched. Google reports the approach cut fabric cost by roughly
30% and fabric power by around 40%, while making topology itself a schedulable resource.
The ML bandwidth explosion
Then machine learning detonated the demand curve.
Scaling laws
reward ever-larger models trained on ever-larger clusters — and a training step is a
synchronised operation: every accelerator must exchange gradients with its peers before
anyone proceeds. All-reduce arithmetic makes the appetite concrete: a 100-billion-parameter model
has ~200 GB of 16-bit gradients per step; a ring all-reduce moves roughly twice that through every
chip's links every step, several steps per second. Multiply out and the cluster's internal
traffic dwarfs its traffic to the outside world — which is why an accelerator now ships with more
link bandwidth than some datacenters had spines:
// All-reduce appetite of one training step (ring algorithm: ~2× data through each chip).
const paramsB = 100e9; // parameters
const bytesPerParam = 2; // bf16 gradients
const gradientsGB = paramsB * bytesPerParam / 1e9;
const perChipGBs = 900; // scale-up link bandwidth per accelerator, GB/s
const stepCommSeconds = 2 * gradientsGB / perChipGBs;
console.log("gradients: " + gradientsGB + " GB; comm per step: " + stepCommSeconds.toFixed(2) + " s");
console.log("at 2 steps/s this must overlap compute -> links must stay saturated");
// The photonic rule of thumb from the ONoC lesson: full pipes favour photons.
const clusterChips = 4096;
const totalTBs = clusterChips * perChipGBs / 1000;
console.log("cluster scale-up bandwidth: " + totalTBs + " TB/s, held busy every step");
Full pipes, long distances, thousands of endpoints: the exact conditions
last
lesson's autopsy said favour photons. And so, at cluster scale, they got them.
Google's TPUv4 supercomputer wires 4,096 chips into a 3-D torus whose inter-cube faces pass
through optical circuit switches: the OCS layer lets schedulers carve out arbitrarily-shaped
slices, re-twist the torus for a job's communication pattern, and route around failed racks by
re-pointing mirrors — topology as software. GPU fabrics are converging on the same physics from
the other side: a scale-up domain like
NVLink
pushes terabytes per second per device, copper caps it at a couple of metres — one rack — and
breaking past that wall means optical I/O on the GPU package: co-packaged engines, dense WDM,
pJ/bit budgets. Every thread of this module, braided into one cable.
Seeing "optical switch" in a datacenter diagram, it is fatally easy to imagine a faster Ethernet
switch. An OCS is a different kind of thing: a motorised patch panel. It makes no
per-packet decisions — it cannot, since (as the
fabrics
lesson established) it can neither buffer a photon nor read a header. While its
mirrors tilt, light on the moving path is simply lost — tens of milliseconds of darkness,
an eternity beside a 100 ns packet time. The deployments succeed because they never ask the OCS to
do a packet switch's job: reconfiguration is scheduled (between training jobs, or as traffic
matrices drift), packet-level statistical multiplexing still happens in the electrical switches at
the edges, and the OCS provides cheap, rate-proof, reconfigurable capacity between them.
Electrons decide; photons carry. The division of labour is the moral of this entire module — an
OCS deployment is simply that moral, bolted into a rack.
There is something philosophically delicious about the Palomar OCS: it is a core component of the
world's most advanced packet networks that contains no bits. No SerDes, no buffers, no
forwarding tables — just 136 beams, an array of two-axis MEMS mirrors a few hundred microns wide,
and a feedback camera watching dim alignment beacons to keep each beam centred on its target
(insertion loss: about 2 dB, a line item any link budget can absorb). Its ancestors were the
telecom optical cross-connects of the dot-com era — built for a bandwidth boom that arrived a
decade late, many of their makers dead by 2003; the technology waited patiently for a customer
whose traffic actually justified it. When the upgrade from 100G to 400G arrived, Google's spine
needed no forklift: the mirrors reflected the new rate as indifferently as the old, because a
mirror does not know what a baud is. In an industry that measures device generations in months,
the most future-proof box in the building turned out to be the one with no electronics in the
datapath at all.
Module 8, closed — and the road on
Tally the module's verdict. Photonics lost the logic war (Module 7) but owns transport: it carries
every long link (WDM
links), is marching into the package
(CPO),
switches wholesale capacity with mirrors
(fabrics,
Apollo), and stops — honestly — where distances shrink below the crossover
(the
ONoC autopsy). Photons carry; electrons compute and buffer. The next module asks the
subversive follow-up: what if the photon's quantum nature — the very thing classical links average
away — were itself the computational resource? That story begins with
photons as
qubits; and for the systems view of how all these links are controlled and kept in
calibration, Module 11's
electronic–photonic
interface awaits.