The Multicore OS and Scalability

A modern server does not have four cores. It has a hundred, or two hundred — an AMD EPYC or an ARM Graviton packs dozens of cores per socket, two sockets to a box. The monolithic kernel you have studied all course — Linux, Windows — is a single shared program in a single shared address space, and every one of those cores runs it. Which raises the master's-level question this lesson exists to answer: does a shared-memory monolithic kernel actually scale to hundreds of cores, or does it hit a wall?

The uncomfortable answer is: not for free, and sometimes not at all. Add a core and, naively, the kernel should do more work per second. But the kernel is shared state — one run queue, one page allocator, one set of locks — and shared state on many cores means contention: cores queueing for the same lock, and, underneath, the cache-coherence fabric grinding as they fight over the same cache lines. Past some core count, adding cores makes the system slower. This lesson is about that wall, Linux's decades-long war against it, and the radical research answer — the multikernel — that says: stop sharing memory, start passing messages.

Three bottlenecks stacked on top of each other

When people say "the kernel doesn't scale" they usually conflate three distinct problems, each below the last:

The deepest of these is the third, and it reframes everything. If a 128-core chip is already a message-passing network wearing a shared-memory costume, maybe the OS should stop pretending otherwise.

Seeing the wall

Plot speedup against core count. The dashed line is the dream: linear, N cores give N\times. The shared-lock curve is Amdahl's law in action — with a serial fraction f, speedup is S(N) = \frac{1}{f + \dfrac{1 - f}{N}} \le \frac{1}{f}. Push the serial-fraction slider up and watch the ceiling drop. The message-passing curve replaces the shared critical section with per-core work coordinated by messages — no single hot cache line — so it keeps climbing.

The gap between the two solid curves at 128 cores is the entire argument for rethinking the kernel. It is not a constant factor you can tune away — it is a different shape.

The multikernel: treat the machine as a network

This is the thesis of Barrelfish (ETH Zürich and Microsoft Research, ~2009). Instead of one kernel whose state all cores share and lock, run one small kernel per core, each with its own private state, and have them coordinate the way distributed machines do — by explicit message passing, not shared memory. Kernel state (the process table, memory allocations) is replicated across cores and kept consistent with an agreement protocol, exactly like a replicated distributed database. The slogan: "the machine is a distributed system; program it like one."

Why would you accept the overhead of messages inside one box? Because on a coherence-bound machine, a message (which you can pipeline, batch, and send asynchronously) can be cheaper than a shared-memory update that stalls a core while a cache line is dragged across the die. It also makes the OS heterogeneity-ready: if some cores are ARM and some are x86, or a core is a GPU or a NIC, there is no shared cache-coherent memory to rely on anyway — messages are the only common language. Related research systems (MIT's Corey, which lets applications declare what they share so the kernel need not assume everything is shared; fos, the "factored OS") chase the same insight from different angles.

Shared lock versus message passing, simulated

Model both designs doing the same total work as cores grow. The shared-lock design serialises its critical sections (Amdahl). The message-passing design does per-core work plus a coordination cost that grows only gently. Watch which one keeps scaling.

// Two kernels doing 1_000_000 units of work, as core count grows. // - shared-lock: a fraction f of every unit is a serial critical section (Amdahl's law). // - message-passing: per-core work, plus a small per-core coordination cost, no serial section. const WORK = 1_000_000; const f = 0.02; // 2% of work is inside the shared critical section function sharedLockTime(cores: number): number { const serial = WORK * f; // must run one-at-a-time const parallel = (WORK * (1 - f)) / cores; return serial + parallel; } function messageTime(cores: number): number { const perCore = WORK / cores; // work splits cleanly const coordination = 40 * cores; // messages to agree; grows gently return perCore + coordination; } const base = sharedLockTime(1); console.log("cores | shared-lock speedup | message-passing speedup"); for (const c of [1, 2, 4, 8, 16, 32, 64, 128, 256]) { const s = (base / sharedLockTime(c)).toFixed(1); const m = (base / messageTime(c)).toFixed(1); console.log(`${String(c).padStart(4)} | ${s.padStart(6)}x | ${m.padStart(6)}x`); } console.log(`\nshared-lock ceiling = 1/f = ${(1 / f).toFixed(0)}x, no matter how many cores`);

Linux's answer: win the war incrementally

Barrelfish is a clean-slate research OS. Linux cannot start over — it must scale the monolith it has, in production, without breaking anything. Its strategy is a decades-long campaign of removing shared state one hot spot at a time, and it has been remarkably successful:

TechniqueIdeaKills which bottleneck
Per-CPU datagive each core its own copy (run queue, allocator freelist); no shared linecache-line + lock contention
RCUreaders never lock or write; writers publish a new versionread-side contention (huge in Linux)
MCS / queued lockseach waiter spins on its own cache line, not a shared onecache-line ping-pong on contended locks
Lock-free / atomicscarefully ordered atomic ops instead of a locklock overhead on the hottest paths
Sloppy / split countersper-core counters summed lazily instead of one shared counterthe coherence traffic of a global counter

The famous MIT study "An Analysis of Linux Scalability to Many Cores" (Boyd-Wickizer et al., 2010) tested exactly this: could stock Linux scale to 48 cores? The finding was encouraging and pointed — with a modest number of targeted patches (mostly the techniques above), Linux scaled well; the bottlenecks were specific shared data structures, not a fundamental flaw in the monolithic design. That result is why Linux, not a multikernel, runs the world's 128-core servers today. The multikernel's ideas, meanwhile, live on wherever shared coherent memory genuinely runs out: across NUMA nodes, across accelerators, and across the machines of a datacenter.

Amdahl's law says extra cores stop helping once the serial part dominates — a plateau. But real many-core kernels sometimes show a curve that actually turns downward past a peak. Why? Because a contended lock is not free even for the core that is only waiting. A naive spinlock has every waiter hammering the same cache line; each acquisition attempt invalidates all the other copies, so the contention itself generates coherence traffic that grows with the number of waiters — O(N^2) messages for N spinning cores in the worst case. Adding a core adds a spinner, which adds traffic, which slows the core that actually holds the lock. That is the collapse queued locks (MCS) were invented to prevent, and it is why "just throw more cores at it" can be worse than useless. Contention isn't a missed opportunity; past a point it is an active tax.

The classic mental error is to treat "all cores share memory" as a free, flat, instantaneous fact — because the programming model presents it that way. It is not. Coherent shared memory is a service the hardware works hard to provide, and it costs real interconnect traffic that scales with the number of cores touching a line. Reading a variable another core just wrote is not a register access; it can be a cache miss serviced across the die, hundreds of cycles. So "just use a shared counter / a shared list" is not a neutral default on a many-core machine — it is a decision to buy coherence traffic. The whole point of per-CPU data, RCU, and the multikernel is to stop paying for sharing you did not actually need. When you design for scale, treat every shared, mutable, hot cache line as a potential bottleneck, and ask whether the cores really need to share it at all.