A modern superscalar core is a wide machine: it can issue four, six, even eight instructions every cycle. But it almost never gets to. Real code is full of stalls — a cache miss here, a dependency chain there, a mispredicted branch — and on any given cycle a huge fraction of those issue slots sit empty, executing nothing. You paid for eight lanes of highway and most of them are bare tarmac. Simultaneous multithreading (SMT) — Intel calls its version Hyper-Threading — is the beautifully simple idea that fills them.
If one thread cannot keep the wide core busy, run two threads on it at once and let the core issue from whichever one has work ready this cycle. The empty slots left by thread A get filled by thread B. No second set of execution units, no second core — just a second architectural state (a second set of registers and program counter) feeding the same back-end. It is one of the highest-leverage tricks in the whole book: a few percent more silicon for a large throughput gain.
Each grid below is one core's issue slots: columns are the issue width (4 wide here), rows are successive clock cycles. A coloured square is an instruction issued; a blank square is a wasted slot. On the left, a single thread leaves most slots empty — the core is starved. On the right, SMT weaves a second thread through the gaps, and utilisation leaps.
Crucially, SMT attacks a different kind of waste than a wider issue width would. Building a wider
core helps only if one thread has enough independent instructions to fill the extra lanes — and we saw with
the
The trick works because most of an out-of-order core is expensive and reusable, while the "identity" of a thread is cheap to replicate.
That last point is the whole trade. SMT is a throughput optimisation. If you care about one latency-critical thread finishing as fast as possible, a second thread sharing its caches and units is a distraction; if you care about total work per second across many threads, SMT is close to free performance.
Let's count exactly what the picture shows: how full the issue slots are with one thread versus two. The model marks each slot with which thread (if any) issued into it and computes utilisation and throughput:
SMT and multicore both give you more threads, but they are opposite bargains. Multicore duplicates the
whole core — execution units and all — so
| SMT (2 threads / core) | 2 cores | |
|---|---|---|
| Execution units | shared (one set) | duplicated (two sets) |
| Caches | shared | private per core (usually) |
| Extra silicon | tiny (~5%) | a whole extra core (~100%) |
| Typical throughput gain | ~15–30% | up to ~100% |
| Helps when | threads stall and leave slots idle | you have genuinely parallel work |
The two compose: a chip with 8 cores and 2-way SMT presents
Because the two threads share the caches. If each thread has a working set that just fits in the L2 cache, running them together on one core means they evict each other's data constantly — cache thrashing — and both slow down more than the extra issue slots gain. This is why HPC and some database and game workloads famously turn Hyper-Threading off: their carefully tuned threads want the whole cache to themselves. SMT is a bet that stalls (idle slots) cost more than the contention it adds — usually true for mixed server workloads, sometimes false for cache-hungry ones. There was even a period where security (the L1TF / MDS side-channel attacks, which leak data between the two shared threads) led cloud providers to disable it entirely.
Your operating system proudly reports "16 logical processors," and it is tempting to treat them as 16
independent cores. They are not. Eight of them are SMT siblings sharing a back-end with the other eight. Pin
a compute-bound job to all 16 and you will not get double the throughput of 8 — you will get maybe