A modern server does not have four cores. It has a hundred, or two hundred — an AMD EPYC or an ARM Graviton packs dozens of cores per socket, two sockets to a box. The monolithic kernel you have studied all course — Linux, Windows — is a single shared program in a single shared address space, and every one of those cores runs it. Which raises the master's-level question this lesson exists to answer: does a shared-memory monolithic kernel actually scale to hundreds of cores, or does it hit a wall?
The uncomfortable answer is: not for free, and sometimes not at all. Add a core and, naively, the kernel
should do more work per second. But the kernel is shared state — one run queue, one page
allocator, one set of locks — and shared state on many cores means contention: cores
queueing for the same lock, and, underneath, the
When people say "the kernel doesn't scale" they usually conflate three distinct problems, each below the last:
The deepest of these is the third, and it reframes everything. If a 128-core chip is already a message-passing network wearing a shared-memory costume, maybe the OS should stop pretending otherwise.
Plot speedup against core count. The dashed line is the dream: linear,
The gap between the two solid curves at 128 cores is the entire argument for rethinking the kernel. It is not a constant factor you can tune away — it is a different shape.
This is the thesis of Barrelfish (ETH Zürich and Microsoft Research, ~2009). Instead of one kernel whose state all cores share and lock, run one small kernel per core, each with its own private state, and have them coordinate the way distributed machines do — by explicit message passing, not shared memory. Kernel state (the process table, memory allocations) is replicated across cores and kept consistent with an agreement protocol, exactly like a replicated distributed database. The slogan: "the machine is a distributed system; program it like one."
Why would you accept the overhead of messages inside one box? Because on a coherence-bound machine, a message (which you can pipeline, batch, and send asynchronously) can be cheaper than a shared-memory update that stalls a core while a cache line is dragged across the die. It also makes the OS heterogeneity-ready: if some cores are ARM and some are x86, or a core is a GPU or a NIC, there is no shared cache-coherent memory to rely on anyway — messages are the only common language. Related research systems (MIT's Corey, which lets applications declare what they share so the kernel need not assume everything is shared; fos, the "factored OS") chase the same insight from different angles.
Model both designs doing the same total work as cores grow. The shared-lock design serialises its critical sections (Amdahl). The message-passing design does per-core work plus a coordination cost that grows only gently. Watch which one keeps scaling.
Barrelfish is a clean-slate research OS. Linux cannot start over — it must scale the monolith it has, in production, without breaking anything. Its strategy is a decades-long campaign of removing shared state one hot spot at a time, and it has been remarkably successful:
| Technique | Idea | Kills which bottleneck |
|---|---|---|
| Per-CPU data | give each core its own copy (run queue, allocator freelist); no shared line | cache-line + lock contention |
| readers never lock or write; writers publish a new version | read-side contention (huge in Linux) | |
| MCS / queued locks | each waiter spins on its own cache line, not a shared one | cache-line ping-pong on contended locks |
| Lock-free / atomics | carefully ordered atomic ops instead of a lock | lock overhead on the hottest paths |
| Sloppy / split counters | per-core counters summed lazily instead of one shared counter | the coherence traffic of a global counter |
The famous MIT study "An Analysis of Linux Scalability to Many Cores" (Boyd-Wickizer et al., 2010) tested exactly this: could stock Linux scale to 48 cores? The finding was encouraging and pointed — with a modest number of targeted patches (mostly the techniques above), Linux scaled well; the bottlenecks were specific shared data structures, not a fundamental flaw in the monolithic design. That result is why Linux, not a multikernel, runs the world's 128-core servers today. The multikernel's ideas, meanwhile, live on wherever shared coherent memory genuinely runs out: across NUMA nodes, across accelerators, and across the machines of a datacenter.
Amdahl's law says extra cores stop helping once the serial part dominates — a plateau. But real
many-core kernels sometimes show a curve that actually turns downward past a peak. Why? Because a
contended lock is not free even for the core that is only waiting. A naive spinlock has every
waiter hammering the same cache line; each acquisition attempt invalidates all the other copies, so the
contention itself generates coherence traffic that grows with the number of waiters —
The classic mental error is to treat "all cores share memory" as a free, flat, instantaneous fact — because the programming model presents it that way. It is not. Coherent shared memory is a service the hardware works hard to provide, and it costs real interconnect traffic that scales with the number of cores touching a line. Reading a variable another core just wrote is not a register access; it can be a cache miss serviced across the die, hundreds of cycles. So "just use a shared counter / a shared list" is not a neutral default on a many-core machine — it is a decision to buy coherence traffic. The whole point of per-CPU data, RCU, and the multikernel is to stop paying for sharing you did not actually need. When you design for scale, treat every shared, mutable, hot cache line as a potential bottleneck, and ask whether the cores really need to share it at all.