Optical Neural Networks

In 2017 a group at MIT pushed laser light through 56 Mach–Zehnder interferometers etched into a silicon chip and used the pattern that came out to recognise spoken vowels. The paper — Shen et al., Deep learning with coherent nanophotonic circuits — is the founding document of the optical neural network (ONN), and its pitch is one line long. A neural network layer computes \mathbf{y} = f(W\mathbf{x}): forward propagation is almost entirely matrix–vector multiplication, and the last module built a machine — the MZI mesh — that performs matrix–vector multiplication passively, in a single transit of light. Encode \mathbf{x} as optical amplitudes, and W\mathbf{x} happens because interference happens: no clock, no multiply units, no fetching of weights from memory. The weights are the chip's phase settings.

That is a genuinely beautiful mapping, and this module spends seven lessons finding out exactly how far it goes. The ethos throughout is the engineer's, not the press office's: we will count every conversion, every calibration loop and every picojoule, because the case for photonic AI hardware stands or falls on arithmetic, not on the undeniable glamour of computing with light.

The Shen architecture: a network as a stack of meshes

One ONN layer is three stages in a row. First, encoding: N modulators put the entries of \mathbf{x} onto N waveguides as optical amplitudes. Second, the mesh: since a lossless mesh can only perform a unitary rotation, an arbitrary weight matrix is factored by the singular value decomposition, W = U \Sigma V^{\dagger} — a mesh for V^{\dagger}, a column of attenuators (or amplifiers) for the diagonal \Sigma, a mesh for U — exactly the SVD construction from Module 4. Third, the activation f, applied channel by channel. Stack layers and you have a deep network in glass and silicon.

Notice what the middle stage costs at run time: nothing per operation. Once the phases are set, an N \times N matvec — N^2 multiplies and N(N-1) adds — is performed by propagation itself, in the time light takes to cross a few millimetres. The two stages either side of it, encoding and activation, are where the trouble lives, and lessons 4–7 are essentially an audit of that trouble.

What maps well — and what doesn't

Be precise about the win. Digital hardware performs an N \times N matvec in O(N^2) multiply–accumulate operations (MACs), each burning energy and each — more importantly — requiring the weight to be fetched. The mesh performs all N^2 MACs concurrently, in one pass, with the weights held in place as phase settings. The advantages compound with N; the overheads (as we will see) are mostly per-vector, not per-MAC. Hence the rule of thumb:

Workload ingredient	Photonic fit	Why
Large dense matvec / matmul	Excellent	N² MACs per pass, passive, weights stationary
Convolutions	Good	Unrolled into matmuls, or done with wavelength tricks (next lesson)
Nonlinear activations	Poor	Photons barely interact — lesson 4 is devoted to this
Weight storage & update	Poor	Phases are set slowly (thermal: microseconds) and hold no dense DRAM-like state
Branching, control, softmax, normalisation	Poor	These stay electronic; every crossing of the boundary costs a conversion

A worked number makes the headline concrete. Take N = 64. The mesh pair plus diagonal needs roughly 2 \cdot \tfrac{N(N-1)}{2} + N \approx 4{,}100 MZIs. At ~100 µm per MZI and mesh depth N, light crosses about 6.4 mm of waveguide; with group index ~4 that is a transit time of \tau \approx 6.4\,\mathrm{mm} \times 4 / c \approx 85\ \mathrm{ps}. Stream input vectors at 10 GHz and the chip sustains 64^2 \times 10^{10} \approx 4 \times 10^{13} MAC/s — about 40 TMAC/s (80 TOPS in marketing units, which count a MAC as two operations) — from a few square millimetres of passive interference. That is the honest headline; the honest footnotes occupy the rest of the module.

The inference–training asymmetry

The mesh has a split personality, and it decides what ONNs are actually for. During inference the weights are frozen: set the phases once — even if setting them takes microseconds and milliwatts of heater power — and then amortise that cost over billions of input vectors streamed through at optical speed. The workload is "same matrix, endless new vectors", which is precisely the mesh's best case, and it is also deep learning's dominant commercial workload: a trained model serving queries. Latency helps too — a forward pass in ~100 ps, with no batching needed to reach peak throughput, is attractive wherever a single answer is needed now (autonomous systems, high-frequency inference, physics triggers).

Training inverts every one of those advantages. Weights change every step, so the slow, power-hungry phase updates land on the critical path; backpropagation needs the transpose pass and stored activations, which the forward-only mesh does not naturally provide; and gradient arithmetic wants more precision than an analog device delivers. Training photonic networks is such a distinct problem that it gets its own lesson. For now, hold the asymmetry: photonics accelerates inference first; training is the hard mode.

The Shen et al. demonstration deserves both its fame and a careful reading. The chip was a 4-input, 4-output mesh of 56 MZIs implementing two network layers; the task was recognising one of four vowel sounds from recorded formant data. Run entirely in simulation, their network scored 91.7%. Run with the matvecs on the physical chip, it scored 76.7%. The gap was not a mystery — the authors traced it to phase-encoding error and photodetection noise, exactly the analog error sources lesson 5 will quantify. And the "deep learning" was modest: the nonlinearity between layers was applied in software, on a computer, between chip passes. None of this is a criticism — a first demonstration is supposed to expose the real problem list — but it set the module's agenda in 2017 and the agenda has not changed since: conversions, nonlinearity, noise, training, energy. Those are the five fences an ONN must clear, and they are the next five lessons in order.

The classic overclaim — in papers as often as press releases — is to demonstrate an optical matvec and describe an optical neural network. Trace one full layer of the 2017 architecture and count the boundary crossings: digital weights → DAC → phase setting; digital inputs → DAC → modulator; light through the mesh (the only optical part); photodetector → ADC → digital nonlinearity → back to a DAC for the next layer. That is an electronic sandwich with an optical filling, and every slice of bread costs energy and time that the interference did not. The per-MAC energy of the optical stage can be genuinely tiny while the per-vector conversion cost dominates the system — whether it does depends on N, and settling that ratio honestly is exactly the ledger of lesson 7. When you read any ONN result, first question: where are the ADCs, and how often does the signal visit them?

Where this goes next

The MZI mesh is the coherent, single-wavelength way to multiply by a matrix — but light has a dimension electronics entirely lacks: colour. The next lesson puts many wavelengths into one waveguide and turns a photodetector into an accumulator, giving a second, incoherent architecture for the same matvec — the wavelength crossbar — and with it the module's recurring trade: coherence versus colour.