A vendor slide says "400 TOPS/W — 100× beyond GPUs." Is that a breakthrough, a rounding convention, or a crime scene? By the end of this lesson you will be able to tell, and quickly. TOPS/W is a fraction, which means there are exactly two ways to flatter it: inflate the numerator (count operations generously) or deflate the denominator (count watts forgetfully) — and photonic systems, with their unusually long tail of off-headline costs, offer more room for forgetfulness than any hardware since the analog computers of the 1950s. None of this requires bad faith; it only requires a boundary drawn where the drawing is easiest. Benchmarking is the craft of redrawing that boundary around the whole machine, normalising for precision, and then asking the question that no peak number answers: can the machine actually be fed? For that last part we will reuse a tool you already own — the roofline model.

The numerator: what is an "op"?

Start with the defensible part. An N \times N mesh clocked at vector rate f performs N^2 multiply-accumulates per pass, and by universal convention one MAC counts as two operations (a multiply and an add):

The mischief starts after that. "Effective" ops: some claims multiply the count by a sparsity factor (ops that were skipped are ops that were "done"), or count a low-precision op as several high-precision ones. Peak vs delivered: 2N^2f assumes every channel busy every cycle; real layers rarely fill a fixed N, and a mesh multiplying a 20\times 20 matrix in a 64-wide engine delivers a tenth of its peak. Precision-free ops: an op at 4 bits is not an op at 8 — digital multiplier energy grows roughly quadratically with bit width, so a fair cross-machine comparison states its precision or normalises to a common one (INT8-equivalent is the customary anchor). The numerator rule: ops must be countable, precision-tagged, and delivered on a named workload.

The denominator: six line items, usually two reported

Now the forgetful half of the fraction. A complete photonic accelerator draws power in six places, and this course has priced every one of them: the laser (at wall-plug efficiency — a 20%-efficient laser turns 1 W of needed light into 5 W at the socket, per the on-chip lasers lesson); the DACs and drivers; the ADCs and TIAs (the interface lesson's exponential price list); the thermal tuning that holds thousands of phases (the drift lesson's 20-watt surprise); and the digital orchestration — SRAM, scheduling, calibration loops. A worked example in round numbers, for the 82-TOPS mesh above:

Brochure arithmetic: 82 / 4 \approx 20 TOPS/W. Honest arithmetic: 82 / 24.2 \approx 3.4 TOPS/W — a factor of six, conjured purely by boundary-drawing, before any question of utilisation. Neither number is fabricated; they simply answer different questions, and only one of them is the question a datacenter operator is asking. The denominator rule: watts are measured at the wall, for the whole machine, while running the named workload.

The roofline: can the beast be fed?

Even an honestly-measured peak means nothing if data cannot arrive fast enough — and here photonics meets an unglamorous truth: its memory system is electronic. Place the accelerator on a roofline: attainable throughput is \min(\text{peak},\, \mathrm{BW}\times I), where I is the workload's arithmetic intensity in ops per byte fetched. A weight-stationary mesh streaming batch-1 vectors achieves I \approx 2 ops/byte on its activations — deep in memory-bound territory — while a well-batched layer with resident weights can reach hundreds. The ridge point I^* = \text{peak}/\mathrm{BW} tells you the minimum reuse a workload must have before the photonic peak is even reachable: raise the peak tenfold without touching the memory system and you have mostly moved the ridge further from your workloads. Slide the peak in the chart and watch how little the left half of the picture — where real inference layers live — changes.

Latency, throughput, and the auditor's checklist

One more axis, then the checklist. Photonic marketing loves latency ("inference at the speed of light") and photonic physics half-supports it: light does cross a mesh in picoseconds. But end-to-end latency includes DAC settling, detector integration, ADC conversion and the digital round trip — typically nanoseconds to microseconds — and quoted TOPS figures usually assume saturating batch sizes that are themselves a latency cost. Throughput at batch 512 and latency at batch 1 are different products; insist on knowing which one is being sold. Here is the full audit, compressed:

• Precision: how many effective bits, verified how — and is the comparison precision-normalised?
• Denominator: laser wall-plug, DAC/ADC, drivers/TIAs, tuning, digital control — all included? Measured at the wall?
• Numerator: peak or delivered? On what named workload, at what batch size and utilisation? Any sparsity or "effective-op" multipliers?
• Accuracy: what did the analog system score on the benchmark, versus the digital baseline — and was the model retrained to survive the hardware?
• Roofline: what memory bandwidth feeds the engine, and what arithmetic intensity does the claimed workload actually have?
• Baseline: compared against which electronic chip, from which year, at which precision?

Six questions, thirty seconds each. Most extraordinary claims do not survive the second one.