Microkernels and Message Passing
The previous
lesson left us with a provocation: a microkernel puts almost nothing in the privileged
kernel. So if the file system, the network stack, and every driver are ordinary user processes, how does
anything get done? A program can't call a function in another address space. The answer is the
one primitive a microkernel really cares about: message passing — inter-process
communication (IPC). In a microkernel, IPC isn't a feature; it is the bloodstream. Every service request,
every reply, every page fault delegated to a user pager travels as a message through the kernel.
This makes IPC performance the number that decides whether a microkernel is a beautiful idea or
a usable system. The story of microkernels is largely the story of that one number going from
embarrassing (Mach, early 1990s) to spectacular (the L4 family), and of a security model — capabilities —
that grew up alongside it. That is this lesson.
The three things a microkernel keeps
Jochen Liedtke's minimality principle is the microkernel's constitution: a concept
is tolerated inside the kernel only if moving it outside would prevent the system from working at all.
Apply that razor and what survives is a startlingly short list:
- Address spaces — the mechanism that isolates one server from another (built on the
MMU).
Can't be outside: it is the isolation.
- Threads and scheduling — someone must dispense the CPU; that requires privilege.
- IPC — the only way isolated address spaces can cooperate. Without it, isolated
servers are just isolated.
Everything else — file systems, drivers, paging policy, the network stack, even much of memory
management — is exiled to user space. The kernel becomes a tiny, auditable arbiter (seL4 is about
10{,}000 lines of C) whose main job is to move messages between servers
quickly and safely.
One round trip, close up
Microkernel IPC is classically synchronous and rendezvous-based: the client
calls, blocks, and is unblocked only when the server replies — like a function call that happens to span
two address spaces. There is no buffering in the kernel; sender and receiver meet, the message is copied
(or, for small messages, passed in registers), and control hands over. Watch the five stages.
Count the crossings: the client's \texttt{Call} is one kernel entry, the
server's \texttt{Reply} is another, and each involves an address-space switch.
That is why a naïve file read in a microkernel costs several times what it does in Linux — and why making
this exact path fast was the L4 project's obsession.
The "microkernels are slow" era — and its end
The reputation was earned honestly. Mach (CMU, 1980s) became the famous
microkernel — it underlies NeXTSTEP and, to this day, the XNU core of macOS and iOS. But Mach's IPC was
heavy: large, general messages, port lookups, and poor cache behaviour meant a round trip cost on the
order of 100 microseconds — thousands of cycles. Systems built by layering a
Unix server on Mach (like early OSF/1) paid that tax constantly and felt sluggish. The lesson the field
drew — "microkernels are inherently slow" — was wrong, but understandable.
Then Jochen Liedtke built L4 and demolished the premise. By ruthless design — messages
small enough to pass in registers, no kernel buffering, a hand-tuned IPC path, careful cache and TLB
behaviour — L4 cut IPC to a few hundred cycles, roughly 10–20\times
faster than Mach. Suddenly a microkernel could be practical. Its descendants — Fiasco, OKL4 (shipped on
billions of phone basebands), and finally seL4 — carried the torch, seL4 adding a
machine-checked proof that the kernel's C code matches its specification.
| Kernel | Era | One-way IPC (order of magnitude) | Claim to fame |
| Mach | late 1980s | \sim 100 µs (thousands of cycles) | the microkernel; lives on in XNU / macOS |
| L4 (original) | 1993–95 | \sim 150–500 cycles | killed the "microkernels are slow" myth |
| seL4 | 2009– | \sim 300 cycles (fastpath) | first formally verified OS kernel |
Modelling the fast path
Why is a synchronous round trip two crossings, and how does batching or an async design change
the arithmetic? The simulation walks a request from client to server and back, tallying the kernel
entries, then compares a Mach-scale cost with an L4-scale cost for a workload of many requests.
// A synchronous IPC round trip and what it costs at Mach vs L4 speeds.
type Stage = { who: string; act: string; kernelEntry: boolean };
const roundTrip: Stage[] = [
{ who: "client", act: "Call(fs, msg)", kernelEntry: true }, // trap in
{ who: "kernel", act: "transfer msg, switch to server", kernelEntry: false },
{ who: "server", act: "handle request", kernelEntry: false },
{ who: "server", act: "Reply(result)", kernelEntry: true }, // trap in
{ who: "kernel", act: "switch back, deliver reply", kernelEntry: false },
];
let crossings = 0;
for (const s of roundTrip) {
crossings += s.kernelEntry ? 1 : 0;
console.log(`${s.who.padEnd(6)} | ${s.act}${s.kernelEntry ? " [kernel entry]" : ""}`);
}
console.log(`\ncrossings per round trip = ${crossings}`);
const N = 100_000; // requests in a busy second
const machCyclesPerCrossing = 3000; // Mach-scale, heavy IPC
const l4CyclesPerCrossing = 300; // L4-scale, register-fast IPC
const mach = N * crossings * machCyclesPerCrossing;
const l4 = N * crossings * l4CyclesPerCrossing;
console.log(`Mach: ${(mach / 1e9).toFixed(1)}e9 cycles for ${N} requests`);
console.log(`L4: ${(l4 / 1e9).toFixed(1)}e9 cycles (${(mach / l4).toFixed(0)}x cheaper)`);
console.log("=> same architecture; the ONLY difference is IPC speed -> viability");
Capabilities — who is allowed to send to whom
If any process could send a message to any server, isolation would be meaningless — a malicious app could
message the disk driver directly and scribble on the raw device. So modern microkernels (seL4, Fuchsia's
Zircon) gate IPC with capabilities: an unforgeable, kernel-protected token that both
names a resource (a specific endpoint, a page, a thread) and conveys the right to use
it. To talk to the file server you must hold a capability to its endpoint; no capability, no
conversation.
- Capabilities live in kernel-protected slots — a process holds handles, never raw pointers, so it
cannot forge or guess one.
- They are the natural fit for message passing: a \texttt{Call} is
"invoke this capability with this message." Access control and addressing become the same act.
- Rights can be delegated and attenuated — a server can hand a client a read-only
capability to a page — which makes least-privilege composition natural. This contrasts with the
ambient-authority ACL model in monolithic Unix, where a process acts with all its user's rights at once.
In 2009 a team at NICTA in Australia did something audacious: they wrote a formal, machine-checked proof,
in the Isabelle/HOL theorem prover, that seL4's \sim 10{,}000 lines of C
refine an abstract specification — meaning the implementation can never deviate from the spec,
so there are no buffer overflows, null dereferences, or unexpected behaviours, ever, for any
input. Later work extended the guarantee down to the compiled binary and up to security properties
(integrity and confidentiality). The proof cost roughly 20 person-years and
ran to hundreds of thousands of lines of proof script — economically insane for a 30-million-line
monolith, but feasible precisely because the microkernel is tiny. It is the sharpest possible
demonstration of the minimality principle's payoff: shrink the trusted core enough and you can prove it
correct.
The single most common misconception in this area is treating microkernel slowness as a law of nature. It
is not. Mach was slow because of specific implementation choices — large general messages, port
indirection, cache-unfriendly layout — not because putting servers in user space is inherently doomed.
Liedtke's L4 ran the exact same architecture 10–20\times
faster by treating IPC as a first-class engineering problem. So when you read "microkernels are slow",
mentally rewrite it as "Mach's IPC was slow." The modern reality is that L4-family IPC costs a few hundred
cycles — comparable to a single monolithic syscall — and the residual overhead comes from the
number of round trips a task needs, not from any per-trip catastrophe. Design to reduce round
trips, not to fear IPC.