Protection and Isolation Models
A modern machine runs your bank's app, a stranger's JavaScript, a background updater, and the kernel
itself — all on the same silicon, at the same time. The
kernel
is the referee, but a referee needs a rulebook: a precise statement of who is
allowed to do what to which resource. This lesson is about that
rulebook — the mathematical object underneath every permission system you have ever met, and the two
rival ways real systems store it.
The vocabulary is deliberately abstract so it covers everything. A subject is an active
entity that does things — a process, a user, a thread. An object is a passive thing that
is acted upon — a file, a socket, a page of memory, another process. A right (or
permission) is a verb the subject may apply to the object — read, write,
execute, send-signal. Protection is nothing more than deciding, for every
(subject, object) pair, which verbs are allowed.
The protection matrix
Lay every subject down the side and every object across the top, and the complete security state of a
system is one giant table — the access-control matrix of Lampson (1971). The cell in row
s, column o holds the set of rights that subject
s holds over object o:
A[s, o] \;\subseteq\; R, \qquad R = \{\,\texttt{read},\ \texttt{write},\ \texttt{exec},\ \texttt{own},\ \dots\}.
"May process s write file o?" is answered by one
lookup: is \texttt{write} \in A[s,o]? Every access decision the OS makes is,
conceptually, exactly this test. The matrix is the ground truth; everything else in this lesson is an
encoding of it that fits in real memory.
And it must be encoded cleverly, because the full matrix is hopeless to store literally. A server with
10{,}000 users and 1{,}000{,}000 files has a matrix
of 10^{10} cells, almost all of them empty. So no real system stores the whole
grid. Instead they store it sliced — and the direction of the slice is the single most important
design decision in access control.
Two ways to slice the same matrix
There are only two natural ways to cut a table into storable pieces: by column, or by row.
- Slice it by column and each object carries the list of who may touch it. That list is
an access-control list (ACL). When you \texttt{ls -l} a
file and see \texttt{rwxr-x---}, you are reading a (compressed) ACL stored
with the file.
- Slice it by row and each subject carries a bundle of tickets, one per object it may
reach. Each ticket is a capability: an unforgeable token that both names an
object and grants specific rights to it. A Unix file descriptor is the classic example — once
you hold \texttt{fd = 3} for an open file, you need not re-prove your
identity to read it; possession is permission.
Watch the same matrix get sliced both ways. The highlighted column is one file's ACL; the highlighted row
is one process's capability list.
ACLs vs capabilities — the trade-offs
The choice of slice direction is not cosmetic; it decides which operations are cheap and which are
agonising. Auditing "who can touch this file?" is trivial with an ACL (read the column) and
brutal with capabilities (search every subject's tickets). Auditing "what can this process touch?"
is the reverse. And revocation — taking a right back — is the hardest problem in
capability systems, because the tickets have already scattered into subjects' hands.
| Question | ACL (column-wise) | Capability (row-wise) |
| Who can access object o? | easy — read o's list | hard — scan all subjects |
| What can subject s access? | hard — scan all objects | easy — read s's tickets |
| Grant a right | edit the object's list | hand over a copy of the ticket |
| Revoke a right | edit the object's list (immediate) | hard — tickets already delegated |
| Check on access | verify caller's identity vs list | just present the ticket (no identity lookup) |
| Delegation to a helper | awkward (confused-deputy risk) | natural — pass the ticket along |
| Real examples | Unix/POSIX perms, NTFS, SELinux labels | fds, seL4, Capsicum, KeyKOS, WebAssembly imports |
Capabilities elegantly dodge the confused-deputy problem — a privileged program tricked
into misusing its authority on a caller's behalf — because the caller must hand over the exact
ticket, so the deputy can never exceed what it was given. This is why capability designs (seL4, Fuchsia's
Zircon, Capsicum on FreeBSD) keep reappearing at the security-critical edge of the field.
The permission check, in code
Both encodings answer the very same question — \texttt{write} \in A[s,o]? — but
arrive at the answer from opposite directions. Here they are side by side: the ACL walks the object's
list looking for the subject; the capability check just asks whether the subject already holds a matching
ticket. Run it.
// The SAME access-control matrix, encoded two ways. Both answer: may `subject` do `right` to `object`?
type Right = "read" | "write" | "exec";
// (1) ACL encoding: each OBJECT stores who may do what.
const acl: Record<string, Record<string, Right[]>> = {
"report.txt": { alice: ["read", "write"], bob: ["read"] },
"keys.pem": { alice: ["read"] },
};
function aclCheck(subject: string, right: Right, object: string): boolean {
const entry = acl[object]?.[subject] ?? []; // find the subject in the object's column
return entry.includes(right);
}
// (2) Capability encoding: each SUBJECT holds unforgeable tickets (object + rights).
interface Cap { object: string; rights: Right[]; }
const bags: Record<string, Cap[]> = {
alice: [ { object: "report.txt", rights: ["read", "write"] }, { object: "keys.pem", rights: ["read"] } ],
bob: [ { object: "report.txt", rights: ["read"] } ],
};
function capCheck(subject: string, right: Right, object: string): boolean {
const ticket = bags[subject]?.find((c) => c.object === object); // find the object in the subject's row
return ticket?.rights.includes(right) ?? false;
}
for (const [s, r, o] of [["alice","write","report.txt"],["bob","write","report.txt"],
["bob","read","keys.pem"],["alice","read","keys.pem"]] as const) {
const a = aclCheck(s, r, o), c = capCheck(s, r, o);
console.log(`${s} ${r} ${o}? ACL=${a} CAP=${c} ${a === c ? "(agree)" : "(!!)"}`);
}
Least privilege — the master principle
Whatever encoding you pick, how much should each cell contain? Saltzer and Schroeder's 1975
answer still governs the field: the principle of least privilege. Every subject should
hold the minimum set of rights needed to do its job, for the shortest time, and nothing
more. A PDF viewer needs to read one file and draw pixels; it has no business opening network sockets or
reading your SSH keys. If it is ever compromised, the blast radius is bounded by exactly the rights it
held at that instant.
Least privilege is why the whole rest of OS security exists: dropping capabilities after start-up,
seccomp syscall filters,
privilege separation, sandboxes. It is also violated by the most common Unix idiom of all — a program
running as \texttt{root} holds every right, so a single bug is total
compromise. The historical fix, breaking a big privileged daemon into a tiny privileged core plus a large
unprivileged worker (as OpenSSH did in 2002), is least privilege applied with a chainsaw.
- every subject runs with the smallest set of rights sufficient for its task;
- rights are held for the shortest time and dropped as soon as they are done with;
- the guarantee it buys: a compromise's damage is bounded by the rights held at that moment,
not by everything the program could ever have been granted.
Who sets the rights? DAC vs MAC
There is a second, orthogonal axis: who is allowed to change the matrix?
- Under Discretionary Access Control (DAC), the owner of an object sets its
permissions at their discretion. Classic Unix is pure DAC: you \texttt{chmod}
your own files, and you may hand any of your rights to anyone. Flexible — but one careless
\texttt{chmod 777}, or one trojaned program running as you, can leak
everything you own.
- Under Mandatory Access Control (MAC), a central, system-wide policy — set by
an administrator, not the file owner — constrains what is allowed, and no subject may relax it,
not even root. The owner's wishes are subordinate to the policy.
Linux ships MAC as a
Linux Security Module (LSM) layered after the ordinary DAC check: a request must
pass the classic permission bits and the MAC policy. The two big implementations are
SELinux and AppArmor. SELinux uses type enforcement:
every process runs in a domain (e.g. \texttt{httpd\_t}) and every
object carries a type (e.g. \texttt{httpd\_sys\_content\_t}), and a
rule base spells out exactly which domains may touch which types. If no rule permits it, it is denied —
default-deny. AppArmor takes a friendlier path: it confines programs by pathname
profiles rather than labels. This is precisely why a web server broken into via a bug still cannot read
\texttt{/etc/shadow}: DAC might allow it after a mistake, but the MAC policy
never will.
Rings and the reference monitor
All of this rests on a hardware foundation you already know: the CPU's protection rings.
x86 defines four (ring 0 = kernel, ring 3 = user; rings 1–2 largely unused; a hypervisor sits in a "ring
−1"). The privilege bit is what makes the whole matrix enforceable rather than merely advisory —
a ring-3 process physically cannot rewrite the page tables or the ACLs that constrain it.
The component that actually performs the check — the code that consults the matrix on every
access — is the reference monitor. Anderson's 1972 report gave it three non-negotiable
properties, and they are the bar every OS security mechanism is measured against:
- Complete mediation — it is invoked on every access to every
object; there is no back door that skips the check;
- Tamperproof — it cannot be modified by the subjects it polices (hence it lives in
the privileged ring, protected by hardware);
- Verifiable — it is small and simple enough to be analysed, tested, or even formally
proved correct (the appeal of a microkernel like seL4, whose kernel is machine-checked).
The trusted computing base (TCB) is the set of all the hardware and software you must
trust for security to hold — the rings, the kernel, the reference monitor. Every idea in this module is,
at heart, an attempt to make that TCB smaller and its guarantees stronger.
Pure capability machines are almost as old as Unix — Cambridge's CAP computer, the Plessey System 250,
later KeyKOS and EROS — and researchers have adored them for fifty years: no ambient authority, no
confused deputies, clean delegation. Yet ACLs conquered the world. Why? Partly inertia and the sheer
gravitational pull of Unix, but partly a genuine mismatch: humans reason naturally about
"who can see my folder?" (an ACL question) and much less naturally about tracking a swarm of
unforgeable tickets and revoking them later. Revocation, in particular, is a capability system's
Achilles' heel — once you have handed out a ticket you cannot easily claw it back. The modern verdict is
that both were right: your laptop's filesystem is ACL-based, but the sandbox around your browser tab, a
WebAssembly module's imports, and the seL4 microkernel are all pure capabilities. The industry quietly
adopted capabilities exactly where least privilege matters most.
A classic misreading is to treat \texttt{rwx} bits as an unbreakable security
boundary. They are discretionary: the file's owner can change them at will, any program running
as you inherits all your authority (ambient authority), and the superuser bypasses the checks
entirely — \texttt{root} reads a \texttt{0600} file
with a shrug. So "I set it to \texttt{600}, it's safe" is only true against
other unprivileged users, and only until some process of yours is compromised. If you need a
boundary that even root and even the file owner cannot cross, you need mandatory access control
(SELinux/AppArmor) or a capability system — a policy nobody can relax at their discretion. DAC bits and
MAC policy live in different columns of the same battlefield; don't mistake one for the other.