The Graduate View of Operating Systems

You already know what an operating system does — it runs your programs, shares out memory, and talks to the disk. This master's course goes underneath that summary and asks the hard questions: how does one machine safely run a hundred untrusting programs at once? How does the kernel stay in control of a CPU that is, most of the time, running someone else's code? What does it cost, in cycles, to cross from a program into the kernel and back? Modern operating systems — Linux, Windows, the seL4 microkernel — are among the most carefully engineered artefacts humans build, and this course is about the ideas and trade-offs inside them.

The whole subject organises around three illusions the OS sells to every process, and one recurring design principle. The illusions are virtualization (each program thinks it has the CPU and all of memory to itself), concurrency (many things appear to happen at once, correctly), and persistence (data outlives any single program and survives a crash). Hold those three words; every module of this course is a deep dive into one of them.

Mechanism versus policy — the master key

The single most useful lens in OS design is the separation of mechanism from policy. A mechanism is the low-level machinery that answers "how is something done?" — how do we switch from one process to another, how do we protect one program's memory from another. A policy answers "which, or when?" — which process runs next, which page gets evicted when memory is full. Good kernels build a fast, general mechanism once, then let a swappable policy sit on top.

The payoff is that you can change the decision-making — a new scheduler, a new page-replacement rule — without rewriting the delicate machinery underneath. Watch for this split in every module: context switching (mechanism) vs the scheduler (policy); address translation (mechanism) vs which page to evict (policy).

The protected boundary

The reason all this is possible is a hardware feature: the CPU runs in one of (at least) two privilege modes. In user mode some instructions are simply forbidden — a program cannot issue I/O directly, reprogram the memory-management unit, or disable interrupts. In kernel mode everything is allowed. Your programs live on top; the kernel lives in the middle, privileged; and the only way up is through a narrow, controlled doorway — the system call. Step through the layers.

Limited direct execution — how the kernel stays in charge

Here is the central tension. For speed, a user program must run directly on the CPU — if the OS interpreted every instruction, everything would crawl. But if the program runs directly, how does the OS ever get the CPU back, or stop the program doing something forbidden? The answer, the beating heart of every OS, is limited direct execution:

So the kernel is not a babysitter watching every step; it is more like a signalman who has wired the tracks so that certain events must route back through the station. Between those events, the program runs unwatched and fast. The simulation below models a timer firing every few ticks and the scheduler stepping in.

// Limited direct execution: processes run directly, but a timer interrupt (every QUANTUM ticks) // returns control to the kernel, which round-robins to the next process. No process can hog the CPU. const procs = ["browser", "compiler", "shell"]; const QUANTUM = 3; // timer fires every 3 ticks const TICKS = 12; let current = 0; // index of the running process let ranFor = 0; // ticks used in this quantum for (let t = 1; t <= TICKS; t++) { console.log(`tick ${t}: running "${procs[current]}" in USER mode`); ranFor++; if (ranFor === QUANTUM) { // Timer interrupt -> trap into the kernel -> scheduler picks the next process. const next = (current + 1) % procs.length; console.log(` ⏰ timer interrupt -> KERNEL: context switch ${procs[current]} -> ${procs[next]}`); current = next; ranFor = 0; } }

A system call looks in C like an ordinary function call — \texttt{read()}, \texttt{write()} — but it is far from ordinary. Crossing into the kernel means switching privilege mode, saving registers, possibly flushing pipeline and predictor state, and jumping through the trap table; the return does it all again. A plain function call is a handful of cycles; a round trip into the kernel is typically hundreds. That gap is why high-perfor­mance systems fight to make fewer crossings — batching many operations per call (\texttt{io\_uring}), caching results in user space, or even running kernel logic without a trap at all. "Avoid the syscall" is a recurring theme you will meet again and again in this course.

A common mental slip is to imagine the kernel as some other machine, always watching, that your programs talk to over a wire. It is not. The kernel is code that runs on the very same CPU cores as your programs — the difference is purely the privilege bit the hardware is in when those instructions execute. When you make a system call, your core stops running your code and starts running the kernel's code, in kernel mode, then switches back. There is no separate "OS processor." This matters because it means kernel time is stolen from your program's time, on your core — every trap, every interrupt, every context switch is overhead paid out of the same cycle budget, which is exactly why the cost of crossing the boundary is such a big deal.

Where this course goes

With the boundary and limited direct execution in hand, the rest of the course descends into each illusion in depth: the system-call interface and kernel architectures first, then advanced scheduling and virtual memory (virtualization), kernel-grade synchronization (concurrency), journaling and log-structured file systems (persistence), and finally virtualization proper, containers, OS security, and where the field is heading.