Microkernels and Message Passing

The previous lesson left us with a provocation: a microkernel puts almost nothing in the privileged kernel. So if the file system, the network stack, and every driver are ordinary user processes, how does anything get done? A program can't call a function in another address space. The answer is the one primitive a microkernel really cares about: message passing — inter-process communication (IPC). In a microkernel, IPC isn't a feature; it is the bloodstream. Every service request, every reply, every page fault delegated to a user pager travels as a message through the kernel.

This makes IPC performance the number that decides whether a microkernel is a beautiful idea or a usable system. The story of microkernels is largely the story of that one number going from embarrassing (Mach, early 1990s) to spectacular (the L4 family), and of a security model — capabilities — that grew up alongside it. That is this lesson.

The three things a microkernel keeps

Jochen Liedtke's minimality principle is the microkernel's constitution: a concept is tolerated inside the kernel only if moving it outside would prevent the system from working at all. Apply that razor and what survives is a startlingly short list:

Everything else — file systems, drivers, paging policy, the network stack, even much of memory management — is exiled to user space. The kernel becomes a tiny, auditable arbiter (seL4 is about 10{,}000 lines of C) whose main job is to move messages between servers quickly and safely.

One round trip, close up

Microkernel IPC is classically synchronous and rendezvous-based: the client calls, blocks, and is unblocked only when the server replies — like a function call that happens to span two address spaces. There is no buffering in the kernel; sender and receiver meet, the message is copied (or, for small messages, passed in registers), and control hands over. Watch the five stages.

Count the crossings: the client's \texttt{Call} is one kernel entry, the server's \texttt{Reply} is another, and each involves an address-space switch. That is why a naïve file read in a microkernel costs several times what it does in Linux — and why making this exact path fast was the L4 project's obsession.

The "microkernels are slow" era — and its end

The reputation was earned honestly. Mach (CMU, 1980s) became the famous microkernel — it underlies NeXTSTEP and, to this day, the XNU core of macOS and iOS. But Mach's IPC was heavy: large, general messages, port lookups, and poor cache behaviour meant a round trip cost on the order of 100 microseconds — thousands of cycles. Systems built by layering a Unix server on Mach (like early OSF/1) paid that tax constantly and felt sluggish. The lesson the field drew — "microkernels are inherently slow" — was wrong, but understandable.

Then Jochen Liedtke built L4 and demolished the premise. By ruthless design — messages small enough to pass in registers, no kernel buffering, a hand-tuned IPC path, careful cache and TLB behaviour — L4 cut IPC to a few hundred cycles, roughly 1020\times faster than Mach. Suddenly a microkernel could be practical. Its descendants — Fiasco, OKL4 (shipped on billions of phone basebands), and finally seL4 — carried the torch, seL4 adding a machine-checked proof that the kernel's C code matches its specification.

KernelEraOne-way IPC (order of magnitude)Claim to fame
Machlate 1980s\sim 100 µs (thousands of cycles)the microkernel; lives on in XNU / macOS
L4 (original)1993–95\sim 150500 cycleskilled the "microkernels are slow" myth
seL42009–\sim 300 cycles (fastpath)first formally verified OS kernel

Modelling the fast path

Why is a synchronous round trip two crossings, and how does batching or an async design change the arithmetic? The simulation walks a request from client to server and back, tallying the kernel entries, then compares a Mach-scale cost with an L4-scale cost for a workload of many requests.

// A synchronous IPC round trip and what it costs at Mach vs L4 speeds. type Stage = { who: string; act: string; kernelEntry: boolean }; const roundTrip: Stage[] = [ { who: "client", act: "Call(fs, msg)", kernelEntry: true }, // trap in { who: "kernel", act: "transfer msg, switch to server", kernelEntry: false }, { who: "server", act: "handle request", kernelEntry: false }, { who: "server", act: "Reply(result)", kernelEntry: true }, // trap in { who: "kernel", act: "switch back, deliver reply", kernelEntry: false }, ]; let crossings = 0; for (const s of roundTrip) { crossings += s.kernelEntry ? 1 : 0; console.log(`${s.who.padEnd(6)} | ${s.act}${s.kernelEntry ? " [kernel entry]" : ""}`); } console.log(`\ncrossings per round trip = ${crossings}`); const N = 100_000; // requests in a busy second const machCyclesPerCrossing = 3000; // Mach-scale, heavy IPC const l4CyclesPerCrossing = 300; // L4-scale, register-fast IPC const mach = N * crossings * machCyclesPerCrossing; const l4 = N * crossings * l4CyclesPerCrossing; console.log(`Mach: ${(mach / 1e9).toFixed(1)}e9 cycles for ${N} requests`); console.log(`L4: ${(l4 / 1e9).toFixed(1)}e9 cycles (${(mach / l4).toFixed(0)}x cheaper)`); console.log("=> same architecture; the ONLY difference is IPC speed -> viability");

Capabilities — who is allowed to send to whom

If any process could send a message to any server, isolation would be meaningless — a malicious app could message the disk driver directly and scribble on the raw device. So modern microkernels (seL4, Fuchsia's Zircon) gate IPC with capabilities: an unforgeable, kernel-protected token that both names a resource (a specific endpoint, a page, a thread) and conveys the right to use it. To talk to the file server you must hold a capability to its endpoint; no capability, no conversation.

In 2009 a team at NICTA in Australia did something audacious: they wrote a formal, machine-checked proof, in the Isabelle/HOL theorem prover, that seL4's \sim 10{,}000 lines of C refine an abstract specification — meaning the implementation can never deviate from the spec, so there are no buffer overflows, null dereferences, or unexpected behaviours, ever, for any input. Later work extended the guarantee down to the compiled binary and up to security properties (integrity and confidentiality). The proof cost roughly 20 person-years and ran to hundreds of thousands of lines of proof script — economically insane for a 30-million-line monolith, but feasible precisely because the microkernel is tiny. It is the sharpest possible demonstration of the minimality principle's payoff: shrink the trusted core enough and you can prove it correct.

The single most common misconception in this area is treating microkernel slowness as a law of nature. It is not. Mach was slow because of specific implementation choices — large general messages, port indirection, cache-unfriendly layout — not because putting servers in user space is inherently doomed. Liedtke's L4 ran the exact same architecture 1020\times faster by treating IPC as a first-class engineering problem. So when you read "microkernels are slow", mentally rewrite it as "Mach's IPC was slow." The modern reality is that L4-family IPC costs a few hundred cycles — comparable to a single monolithic syscall — and the residual overhead comes from the number of round trips a task needs, not from any per-trip catastrophe. Design to reduce round trips, not to fear IPC.