In January 2018 the entire industry learned that a performance trick baked into every fast CPU for two decades had quietly broken the most fundamental promise an operating system makes. You already met the hardware mechanism in Spectre and Meltdown: the CPU speculates past a branch or a permission check, executes instructions it may have to throw away, and — crucially — leaves microarchitectural footprints (cache-line residency) that a patient attacker can measure. This lesson is the other half of that story: what the operating system had to do about it, why the fix was so drastic, and what it cost.

The reason OS people panicked is specific. Meltdown let an ordinary user process read kernel memory — not through a bug in the kernel, but by exploiting the fact that the kernel was mapped into every process's address space for speed. The isolation the OS thought the hardware was enforcing was, under speculation, not enforced at all. The kernel's own defence — never mapping user pages executable, checking every ACL — was irrelevant, because the leak happened below the architectural level the OS controls.

The assumption that broke

For decades, kernels made a deliberate trade. The kernel's pages were mapped into every process's page tables — marked supervisor-only, so a ring-3 access would fault — precisely so that a system call could enter the kernel without switching page tables. No \texttt{CR3} reload, no TLB flush: the trap just raised privilege and the already-mapped kernel was right there. Fast, and — everyone believed — safe, because the supervisor bit would stop any user read.

Meltdown demolished the "safe" half. On the vulnerable cores, a speculative load of a kernel address returned its value into the pipeline before the permission check retired and squashed the instruction. The value never became architecturally visible — but the attacker had already used it to index an array, pulling one cache line into cache. A timing probe afterwards revealed which line, and thus the secret byte. The permission check was too late. The kernel being mapped for speed had become the kernel being readable for free.

The fix: Kernel Page-Table Isolation (KPTI)

If the danger is that the kernel is mapped while user code runs, the fix is blunt: don't map it. KPTI (Kernel Page-Table Isolation — Linux shipped it as an emergency patch, originally the aptly-named KAISER) gives each process two page tables. The user table maps all the user pages but only a bare-minimum trampoline of the kernel — just the entry/exit stubs needed to make the switch, containing no secrets. The kernel table maps everything, as before. Now, when Meltdown speculates a read of a kernel address from user mode, there is simply nothing mapped there to leak.

The price is paid on every boundary crossing. Where a syscall used to enter the kernel with the same \texttt{CR3} (the register holding the page-table base), it must now reload \texttt{CR3} to swap to the kernel table on entry, and reload it again to swap back on return. A \texttt{CR3} write historically flushes the TLB — throwing away the cached virtual→physical translations you spent cycles filling — so every syscall now also eats a burst of TLB misses as translations are re-walked. Two page tables, switched twice per crossing: that is the KPTI tax.

PCID: taking some of the tax back

The saving grace is a hardware feature that suddenly became essential: PCID (Process-Context IDentifiers, ARM calls the equivalent ASIDs). PCID tags each TLB entry with an address-space identifier, so a \texttt{CR3} reload can switch address spaces without flushing the TLB — the stale entries simply become invisible to the new context and are evicted lazily. With PCID, KPTI's forced \texttt{CR3} switches keep the expensive TLB contents alive across the crossing, turning a full flush into a cheap tag change.

This is why the measured overhead of KPTI varies so wildly in the literature — anywhere from \sim 1\% to \sim 30\%. It depends almost entirely on two things: how syscall-heavy the workload is (a CPU-bound number-cruncher barely crosses the boundary and barely notices; a database or web server doing millions of tiny I/O syscalls per second is hammered), and whether PCID is available to blunt the TLB cost. The general shape: overhead grows with the crossing rate.

The other family: Spectre and retpolines

Meltdown was the easy one — KPTI is a clean structural fix, and newer CPUs fixed the root cause in silicon. Spectre is the stubborn one, because it abuses legitimate speculation. Spectre variant 2 (branch-target injection) lets an attacker train the CPU's indirect-branch predictor so that a victim's indirect jump speculatively lands on an attacker-chosen gadget, which then leaks data through the same cache channel. There is no single page-table trick to fix this; the mitigations are a grab-bag:

• Retpolines ("return trampolines"): the compiler rewrites every indirect call/jump into a construct built from \texttt{ret} that deliberately steers any speculation into a harmless infinite-loop "capture" spin, denying the attacker a useful speculative target. It trades the vulnerable indirect branch for a safe (if slower) return-based one.
• IBRS / IBPB / STIBP: microcode-provided barriers that restrict or flush the branch-predictor state across privilege and context boundaries (the kernel issues an \texttt{IBPB} on context switch so one process cannot poison another's predictions).
• Speculation barriers (\texttt{lfence}) inserted after bounds checks to stop variant-1 (bounds-check bypass) speculating past the check.

Each of these adds cycles to the very hottest paths in the machine — branches and boundary crossings — which is why the 2018 mitigations, taken together, are remembered as one of the largest across-the-board performance regressions in computing history.

Counting the tax, in code

The KPTI cost is concrete: extra \texttt{CR3} reloads, and — without PCID — the TLB refills they trigger. This sim counts them for a workload of N syscalls, with and without PCID, and turns the cost into a rough slowdown. Change the numbers and watch a syscall-heavy server suffer while a compute-bound job shrugs.