eBPF and Kernel Extensibility
Here is a wish that has haunted operating systems since the beginning: I want to add my own code to
the running kernel — a new packet filter, a custom tracer, a security check — without rebooting, and
without the machine crashing if I get it wrong. The kernel is the most privileged, most trusted code
on the machine; it runs in
ring 0
with no seatbelt. Historically your only options were both bad: rebuild the kernel from source (slow,
static), or load a kernel module — native code injected straight into the kernel's
address space, where a single stray pointer takes down the whole machine.
eBPF (the "extended Berkeley Packet Filter") is the modern answer, and it is one of the
most quietly revolutionary things to happen to Linux in a decade. The trick is audacious: let users ship
code into the kernel, but first prove — statically, before it ever runs — that the code
cannot crash the kernel, cannot loop forever, and cannot touch memory it shouldn't. A program
that fails the proof is simply rejected at load time. Code that passes runs at native speed, JIT-compiled,
stitched onto a hook deep in the kernel. This lesson is about how that bargain works and why it changed
systems programming.
The old bargain: power you could not survive
A traditional loadable kernel module (a Linux \texttt{.ko}
file) is compiled native code that \texttt{insmod} drops directly into kernel
space. It is maximally powerful — it can do anything the kernel can — and that is exactly the
problem. There is no sandbox. A null dereference in your module is a null dereference in the kernel: a
kernel panic, a frozen machine, possibly corrupted data on disk. Writing one correctly demands deep
expertise, and even experts ship bugs. You would never let an application developer, let alone an
end user, load one on a production fleet.
So the field lived with a painful trade-off. If you wanted to observe or extend kernel behaviour, you
either paid the cost of crossing into user space for every event (slow), or you accepted the terrifying
risk of native kernel code (dangerous). eBPF dissolves the trade-off by adding a third thing between the
two: a restricted, verifiable execution environment inside the kernel.
- A tiny in-kernel virtual machine. eBPF programs are bytecode for a small RISC-like
VM with ~11 registers and a fixed instruction set — not arbitrary native code.
- A static verifier gates every load. Before a program runs, the kernel walks all of
its execution paths and proves it terminates and only touches memory it is allowed to. No
proof, no load.
- Then JIT to native speed. A verified program is compiled to native machine code, so
it runs as fast as a module — the safety cost is paid once, at load time, not per instruction.
- Attached to hooks. The verified program is hung on a specific kernel event —
a syscall entry, a tracepoint, a network receive path, a scheduler event — and fires there.
The pipeline: verify, then JIT, then attach
Follow a program from your editor to a live kernel hook. You write it in a restricted C (or a one-liner in
\texttt{bpftrace}), compile it to BPF bytecode, and hand it to the kernel via
the \texttt{bpf()} system call. The verifier is the gate: it
performs a symbolic walk of every path through your program's control-flow graph, tracking the possible
range of every register, and rejects anything it cannot prove safe. Only then does the JIT turn
the bytecode into native code and the loader attach it to a hook.
Notice where the cost lands. All the checking happens once, at load — the hot path (the program
firing on every packet or every syscall) is native code with no runtime safety tax. That is the whole
point: move the safety check out of the hot path and into a one-time proof.
What the verifier actually proves
The verifier is not magic and it is not a general theorem prover — it is a deliberately conservative
static analysis, and understanding its rules explains why eBPF programs look the way they do:
- Termination. Historically, no backward jumps at all — the program had to be a DAG, so
it provably halts. Modern kernels allow bounded loops the verifier can prove finite
(an unbounded \texttt{while(1)} is still rejected). This is why there is a
hard cap on total instructions checked (originally 4096, now up to ~1 million).
- Memory safety. Every pointer access must be provably in-bounds. Before you read a
network packet byte you must first compare against the packet-end pointer; the verifier tracks that you
did, and rejects the access otherwise. You cannot fabricate a pointer from an integer.
- Restricted capability. A program may only call a fixed allow-list of
helper functions the kernel exposes (look up a map entry, read a timestamp, redirect a
packet). It cannot call arbitrary kernel functions.
The escape hatch for keeping state and talking to user space is BPF maps: kernel-resident
key/value tables that a program reads and writes, and that a user-space process can also read. That is how
a tracer accumulates a histogram in the kernel and your terminal prints it.
A verifier, in miniature
The real verifier is thousands of lines of careful range-tracking. But its spirit is easy to
model: reject unbounded loops, reject out-of-bounds memory, reject disallowed calls; accept everything
else. The simulation runs three candidate programs through a toy verifier and shows which load and which
bounce.
// A toy eBPF verifier. Each "program" is a list of ops. The verifier proves:
// (1) no unbounded loop, (2) every memory access is in-bounds, (3) only allowed helpers are called.
type Op =
| { kind: "load"; offset: number } // read packet[offset]
| { kind: "helper"; name: string } // call a kernel helper
| { kind: "loop"; bounded: boolean }; // a loop the verifier can/can't bound
const PACKET_LEN = 64; // provable packet size
const ALLOWED = new Set(["map_lookup", "ktime", "redirect"]);
function verify(name: string, prog: Op[]): boolean {
for (const op of prog) {
if (op.kind === "loop" && !op.bounded) {
console.log(` REJECT ${name}: unbounded loop -> cannot prove termination`);
return false;
}
if (op.kind === "load" && (op.offset < 0 || op.offset >= PACKET_LEN)) {
console.log(` REJECT ${name}: load at offset ${op.offset} is out of bounds [0,${PACKET_LEN})`);
return false;
}
if (op.kind === "helper" && !ALLOWED.has(op.name)) {
console.log(` REJECT ${name}: helper "${op.name}" is not on the allow-list`);
return false;
}
}
console.log(` ACCEPT ${name}: proof holds -> JIT and attach`);
return true;
}
verify("packet_counter", [{ kind: "load", offset: 12 }, { kind: "helper", name: "map_lookup" }]);
verify("spin_forever", [{ kind: "loop", bounded: false }]);
verify("wild_read", [{ kind: "load", offset: 999 }]);
verify("bad_call", [{ kind: "helper", name: "format_disk" }]);
Where eBPF lives now — three worlds
Once you can safely run code at kernel hooks, whole categories of tooling collapse into it. Three
ecosystems dominate:
| Domain | Hook it uses | Real tools | What it replaces |
|---|
| Observability | kprobes, tracepoints, USDT | \texttt{bpftrace}, \texttt{bcc}, Pixie | strace/printk hacks; per-event user-space crossings |
| Networking | XDP (at the driver), tc, sockets | Cilium, Katran (Facebook LB) | iptables chains; kernel-bypass complexity |
| Security | LSM hooks, syscall entry | Falco, Tetragon, seccomp-bpf | bolted-on scanners; auditd |
The networking case is the most dramatic. XDP (eXpress Data Path) runs your eBPF program
in the network driver, on the raw packet, before the kernel even builds its socket buffer — so a
DDoS-dropping or load-balancing decision happens at the earliest possible instant, tens of millions of
packets per second per core. Facebook's Katran load balancer and the
Cilium project (the default networking layer for much of Kubernetes) are built on exactly
this. Meanwhile \texttt{bpftrace} lets you answer questions like "which
processes are opening which files, histogram the latency" in a single line, on a live production box, with
no reboot and near-zero overhead.
The name is a fossil. In 1992 the original BPF was a genuinely small idea from Berkeley:
a tiny bytecode language so that \texttt{tcpdump} could push a packet-matching
rule into the kernel and avoid copying every packet up to user space just to throw most away.
That is "classic BPF" — a packet filter, nothing more. Around 2014, Alexei Starovoitov and Daniel Borkmann
rebuilt it as eBPF: more registers, 64-bit, maps, helper functions, a real verifier, and
crucially the ability to attach to any hook, not just the network. The "packet filter" heritage
stuck to the name the way "dialling" stuck to phones. Today eBPF is closer to "a safe, universal, in-kernel
scripting runtime" — some people only half-jokingly call it a JavaScript-for-the-kernel — and the acronym
is mostly retired; Linux folk just say "BPF".
The verifier proves a narrow, precise property: your program will not crash the kernel, loop forever, or
read out of bounds. It does not prove your program is correct — a verified tracer can
still compute a wrong histogram — and it does not mean the program is harmless. A verified,
perfectly "safe" eBPF program with the right privileges can still read sensitive data flowing through the
kernel, mislead an admin, or act as a rootkit hiding processes. That is why loading BPF generally requires
\texttt{CAP\_BPF}/root, why the verifier itself is a juicy attack surface (a
verifier bug that lets an unsafe program slip through is a kernel exploit), and why "unprivileged
eBPF" has been progressively locked down. Read "verified" as "cannot corrupt the kernel by construction",
not as "audited, correct, and benign".
Why this is a big deal
Step back and see the shape of it. For fifty years, extending the kernel meant either recompiling it or
risking it. eBPF adds a genuinely new option to operating-system design: a safe, fast, dynamic
extension mechanism, where safety is established by proof rather than by trust or by isolation.
It moves Linux part-way toward an idea microkernels chased from the other direction — user-controlled
policy without user-controlled crashes — but achieves it inside a monolithic kernel. The next
lessons in this module look at other frontiers where the classic OS is being rethought; eBPF is the one
that already shipped to billions of machines.