SIMD Multimedia Extensions

The vector supercomputer was a whole machine built around data parallelism. In the 1990s a cheaper idea took over the world: don't build a special computer — just bolt a little bit of vector power onto the ordinary CPU everyone already owns. That is packed SIMD — "single instruction, multiple data" — and it is the reason your phone can decode video, your laptop can blur a photo in real time, and a modern server can run a neural network without a GPU.

The trick is delightfully simple. A CPU already has 64-bit registers. A video pixel is only 8 bits. So pack eight pixels into one 64-bit register and add another eight in a single instruction, with the carry between neighbouring bytes cut so they don't bleed into each other. One instruction, eight additions. That's a 8\times speedup on exactly the kind of repetitive byte-crunching that multimedia is made of — for the cost of a handful of new opcodes.

A packed register is a register cut into lanes

Think of a wide register as a tray divided into equal slots. A 128-bit SSE register can be sliced as sixteen 8-bit bytes, eight 16-bit shorts, four 32-bit floats, or two 64-bit doubles — you pick the lane width per instruction. A packed add then adds slot-by-slot, in parallel, with no carry crossing the slot boundaries. The same silicon, re-partitioned, does 16 tiny adds or 2 big ones.

\underbrace{[\,a_0\;a_1\;a_2\;a_3\,]}_{\text{4 floats}} \;+\; \underbrace{[\,b_0\;b_1\;b_2\;b_3\,]}_{\text{4 floats}} \;=\; [\,a_0{+}b_0\;\;a_1{+}b_1\;\;a_2{+}b_2\;\;a_3{+}b_3\,]

This differs from a classic vector machine in one crucial way: the width is fixed in the instruction. "Add four floats" is a different opcode from "add eight floats." There is no vector length register saying "do 22 this time." That fixed width is packed SIMD's original sin — and the reason its history is a story of the register doubling, and doubling, and doubling again.

The width-doubling arms race

Intel's line tells the whole story. MMX (1997) reused the 64-bit floating-point registers for eight-byte integer ops. SSE (1999) added real 128-bit registers and floating point. AVX (2011) doubled them to 256 bits, and AVX-512 (2016) to 512. Every doubling roughly doubles peak throughput — but each is a new instruction set that old binaries can't use until they're recompiled. On the ARM side the same pressure produced NEON (128-bit, fixed) and then, learning the lesson, SVE — Scalable Vector Extension — which finally brought the classic vector machine's length-agnostic style back, so one binary runs on 128-bit or 2048-bit hardware unchanged.

Two ways to actually use it: intrinsics vs auto-vectorization

Nobody wants to hand-write assembly. There are two civilised routes. Auto-vectorization is the compiler spotting a plain loop and quietly emitting packed instructions — free when it works, but it gives up the moment the loop looks scary (pointer aliasing, data-dependent branches, non-unit stride). Intrinsics are C-callable functions that map one-to-one onto SIMD instructions — \texttt{\_mm256\_add\_ps} is "add eight floats" — giving you assembly-level control while the compiler still allocates registers. Numerical and media libraries are full of hand-written intrinsics for exactly the hot kernels the compiler couldn't crack.

The model below simulates a packed add at whatever lane width you choose, and reports how many scalar additions one packed instruction stands in for:

// Simulate a packed-SIMD add: split a wide register into `lanes` equal slots and add slot-by-slot. function packedAdd(a: number[], b: number[]): number[] { const out: number[] = []; for (let i = 0; i < a.length; i++) out.push(a[i] + b[i]); return out; // one packed instruction; the loop here is just the simulation } // 256-bit AVX register as 8 lanes of 32-bit float. const laneWidthBits = 32; const regBits = 256; const lanes = regBits / laneWidthBits; const a = [10, 20, 30, 40, 50, 60, 70, 80]; const b = [1, 2, 3, 4, 5, 6, 7, 8]; console.log(`${regBits}-bit register = ${lanes} lanes of ${laneWidthBits}-bit`); console.log(`packed add result: [${packedAdd(a, b).join(", ")}]`); console.log(`one _mm256_add_ps does the work of ${lanes} scalar adds`); // Same silicon, re-sliced as 16-bit: twice the lanes. const narrowLanes = regBits / 16; console.log(`re-sliced to 16-bit lanes: ${narrowLanes} adds per instruction`);

Masking: how SIMD survives an "if"

SIMD's weakness is branches: all lanes execute the same instruction, so what happens when only some elements should take the "then" branch? The answer is predication / masking. A mask register holds one bit per lane; a masked instruction computes on all lanes but only writes back the lanes whose mask bit is set. To do \texttt{if (x>0) y = x} across a vector, you build the mask x>0 and do a masked store. AVX-512 and ARM SVE make masking first-class — every arithmetic instruction can take a mask — which is what lets modern SIMD vectorize loops that a 1999 SSE machine would have had to run scalar.

Why multimedia and ML love it

The kernels that dominate media and machine learning are perfect SIMD food: apply the same cheap operation to a long, regular array with no data-dependent control flow — brighten every pixel, multiply-accumulate every weight, add every sample. A dot product is a chain of fused multiply-adds; an image filter is a stencil over a grid; a matrix multiply is a mountain of the same. For these, packed SIMD turns one scalar op into 8, 16, or (with AVX-512) 32 at a stroke, which is why the vector units in commodity CPUs quietly do a huge fraction of the world's numerical work — and why AVX-512 added special instructions just for neural-network integer math.

Wide SIMD isn't free. Firing up the 512-bit units draws so much power that early Intel chips lowered their clock frequency whenever AVX-512 code ran — sometimes slowing down all the other, non-vector code sharing the core. So a library sprinkling a little AVX-512 into an otherwise scalar program could make the whole thing slower. Torvalds' 2020 rant captured a real tension: peak throughput and everyday latency pull against each other, and a feature that wins a benchmark can lose the real workload. Later chips softened the frequency penalty, but the lesson stands — more lanes only help if you can keep them fed and afford the watts.

A classic vector machine's code is width-agnostic: the vector length register adapts at runtime. Packed SIMD is the opposite — "add 4 floats" and "add 8 floats" are different instructions. So every widening (128→256→512) forced a new instruction set and a recompile, and a binary built for AVX crashes with an illegal-instruction fault on a CPU that only speaks SSE. This is exactly the design flaw ARM's SVE and RISC-V's V extension fix by resurrecting the vector-length register. Don't assume "it's SIMD, it'll just run wider" — with packed SIMD, wider means a whole new target.