So far every instruction we have met does one thing to one pair of operands:
add x3, x1, x2 adds a single number to a single number. But an enormous share of real computing —
graphics, audio, machine learning, physics, image filters — does the same operation to a whole
array of numbers. Feeding those through one-at-a-time scalar instructions wastes the chip. The answer,
baked into the
Consider adding two arrays element by element,
For a million-element array that is a million trips through the loop. Yet each add is completely independent of the others — nothing stops us doing several at once. That untapped independence is called data-level parallelism, and SIMD is the ISA's way of exposing it.
A SIMD ISA adds wide registers that hold not one number but a packed vector of several. A 128-bit register can hold four 32-bit floats; the hardware slices it into four independent lanes and an add instruction adds all four pairs simultaneously, in the same time a scalar add takes one pair.
This is pure Single-Instruction-Multiple-Data: one opcode, multiple data elements. The width — how many lanes — is a defining property of the extension. Widen the register and you widen the throughput, at least until memory bandwidth or the supply of parallel work runs out.
Almost every mainstream ISA has bolted SIMD on, growing the vector width over the years:
| Extension | ISA | Vector width | Style |
|---|---|---|---|
| SSE | x86 | 128-bit (4 floats) | fixed width |
| AVX / AVX2 | x86 | 256-bit (8 floats) | fixed width |
| AVX-512 | x86 | 512-bit (16 floats) | fixed width |
| NEON | ARM | 128-bit | fixed width |
| SVE / RVV | ARM / RISC-V | agnostic | vector-length agnostic |
Note the split. Older SIMD (SSE, AVX, NEON) hard-codes the width into the opcode — an AVX add is always 8 floats, so binaries must be recompiled to use a wider machine. The modern vector designs (ARM's SVE, RISC-V's RVV) are vector-length agnostic: the program asks the hardware "how long is your vector?" at run time and loops in chunks of that length, so the same binary runs on a 128-bit chip or a 1024-bit one and gets faster automatically. That is a genuinely different, and cleaner, ISA contract.
Here is the crux, in code. The scalar version adds one element per step; the "SIMD" version processes four lanes per step (unrolled to make the parallelism visible), so it needs a quarter of the iterations. On real hardware those four lane-adds happen in a single vector instruction — here we simulate the lanes with an inner loop and count how many vector operations each approach costs.
Four times fewer add-instructions, and the same fourfold cut in loop overhead. The
You rarely write SIMD by hand. Instead the compiler auto-vectorises: it recognises a loop whose
iterations are independent — no iteration reads a value a later one writes — and rewrites it to process
a[i] = a[i-1] + 1, a function call it can't see into, or possible pointer aliasing
(two pointers that might reference overlapping memory) all force it to fall back to safe scalar code. Writing
vectorisable loops — simple, independent, alias-free — is a real skill precisely because the ISA can only exploit
parallelism the compiler can see.
You hit three walls. First, memory bandwidth: a vector unit that can add 64 floats per cycle is useless if memory can only deliver 8 — you starve. Second, diminishing parallel work: Amdahl's law bites, since real loops have serial parts and awkward tail elements, and very wide vectors waste lanes on short arrays. Third — the reason AVX-512 was controversial — power and heat: firing up a huge 512-bit unit drew so much current that some Intel chips lowered their clock speed whenever AVX-512 ran, so code using it could paradoxically slow down neighbouring scalar code. Width is not free; it trades against bandwidth, utilisation and watts. The vector-length-agnostic designs are partly a reaction to this: let the hardware pick a width it can actually sustain.
Three different kinds of parallelism get muddled. SIMD is data parallelism within a single core: one instruction, one thread, many data lanes. Multicore / multithreading is task parallelism across cores: many instruction streams running at once. And pipelining overlaps stages of consecutive instructions. They are complementary — a modern chip uses all three at once — but they solve different problems and live at different ISA/microarchitecture layers. Saying "I made it parallel with AVX" is not the same as "I made it parallel with threads"; conflating them will send you optimising the wrong bottleneck.
SIMD is a bargain, but not a free one, and the cost lands in the contract. Adding vectors means new architectural state (wide vector registers the OS must save and restore on a context switch), new instructions and encodings, rules for masking and tail handling, and — for fixed-width designs — a versioning headache as each new width (SSE → AVX → AVX-512) fragments the software ecosystem. That is the deep lesson of this whole module: the ISA is a contract, and every capability you expose to software — a register model, an addressing mode, a calling convention, a vector unit — is a promise you must keep forever. Choosing what to put in the contract, and what to leave to the compiler and microarchitecture, is the art of computer architecture. Vector and SIMD ideas carry straight into the data-parallelism and GPU material ahead, where these same lanes reappear by the thousand.