The System-Call Interface

In your C program you write \texttt{n = read(fd, buf, 4096)} and it looks exactly like calling \texttt{sqrt} or a helper you wrote yourself. It is not. A plain function call stays inside your process, costs a few cycles, and never leaves user mode. \texttt{read} does something categorically different: it crosses the privilege boundary into the kernel, because only the kernel is allowed to touch a disk, a socket, or another process's pages. That crossing is the system call — the single narrow doorway between the billions of lines of unprivileged code the world runs and the few million lines that are actually in charge of the machine.

You met this boundary in the graduate view as the idea of limited direct execution. This lesson opens the doorway and looks at its hinges: the trap that crosses it, the trap table that decides where control lands, the ABI that says how arguments are passed, and the surprising cost — hundreds of cycles — that makes "avoid the syscall" a design mantra.

A trap, not a jump

A user program cannot simply \texttt{jmp} to a kernel function. If it could, isolation would be a fiction — any process could jump into the middle of the scheduler and do as it pleased. Instead the hardware provides a synchronous trap: a special instruction that atomically (a) raises the privilege level to ring 0, (b) switches to a trusted kernel stack, and (c) transfers control to one fixed entry point the kernel registered at boot. The program never chooses where in the kernel it lands — it only asks, and the kernel's entry code dispatches.

On modern x86-64 the instruction is \texttt{syscall}; the entry point is whatever address the kernel wrote into the \texttt{LSTAR} model-specific register at boot.
The classic 32-bit Linux path was \texttt{int 0x80} — a software interrupt vectored through entry 0x80 of the CPU's interrupt descriptor table (IDT).
On ARM64 it is the \texttt{svc} ("supervisor call") instruction, trapping to the exception vector table.

All three are the same idea wearing different names: a deliberate, hardware-mediated exception that the program raises on itself to request a service. The kernel-side entry point is the top of the trap table — the array of handlers indexed by a number the caller supplies.

The shape of a crossing

Follow a single \texttt{read} from user space, into the kernel, and back. The vertical line is the privilege boundary; nothing crosses it except through the trap.

Notice how little the program did: it loaded a number and some arguments into registers and ran one instruction. Everything else — the mode switch, the stack switch, the dispatch, the return — is machinery the hardware and kernel provide. That is the whole point of a mechanism: build the expensive, delicate crossing once, and expose it as a single instruction.

The calling convention — the syscall ABI

Because the caller and the kernel are compiled separately (often years apart), they must agree on an exact, frozen contract for how the request is encoded: which register holds the call number, which hold the arguments, where the result comes back. This is the system-call ABI, and on Linux/x86-64 it is set in stone — breaking it would break every binary ever compiled.

Role	x86-64 Linux	Notes
syscall number	\texttt{rax}	e.g. 0 = read, 1 = write, 60 = exit
arg 1	\texttt{rdi}	note: user ABI arg 4 is \texttt{rcx}, but…
arg 2	\texttt{rsi}
arg 3	\texttt{rdx}
arg 4	\texttt{r10}	…\texttt{r10} replaces \texttt{rcx}, which \texttt{syscall} clobbers
arg 5	\texttt{r8}
arg 6	\texttt{r9}	six register args max — no stack args
return value	\texttt{rax}	errors come back as -4095 \le \texttt{rax} \le -1 (a negative \texttt{errno})

The \texttt{read()} you call in C is not the syscall — it is a thin wrapper in the C library that marshals its arguments into these registers, executes \texttt{syscall}, and translates a negative return into -1 plus a set \texttt{errno}. The kernel never sees your language; it sees registers.

Dispatch through the trap table

Once inside, the kernel's entry stub does the least work it can — saves the user registers, then uses the number in \texttt{rax} as an index into a fixed array of function pointers (Linux calls it \texttt{sys\_call\_table}). A bounds check rejects out-of-range numbers; a valid one is a single indexed indirect call. Here is that dispatch, in miniature.

// A trap table in miniature: the kernel indexes an array of handlers by the syscall // number the user left in "rax". Out-of-range numbers are rejected, not obeyed. type Handler = (a: number, b: number, c: number) => number; const NR_READ = 0, NR_WRITE = 1, NR_GETPID = 39, NR_EXIT = 60; // The trap table: number -> handler. Gaps are unimplemented calls. const trapTable: Record<number, Handler> = { [NR_READ]: (fd, _buf, n) => { console.log(` sys_read(fd=${fd}, ${n} bytes)`); return n; }, [NR_WRITE]: (fd, _buf, n) => { console.log(` sys_write(fd=${fd}, ${n} bytes)`); return n; }, [NR_GETPID]: () => { console.log(` sys_getpid()`); return 4242; }, [NR_EXIT]: (code) => { console.log(` sys_exit(${code})`); return 0; }, }; function trap(nr: number, a = 0, b = 0, c = 0): number { const h = trapTable[nr]; if (!h) { console.log(` #GP: syscall ${nr} not implemented -> return -ENOSYS`); return -38; } return h(a, b, c); } // A tiny program's stream of syscalls (the number is what really matters): console.log("rax=0 ->", trap(NR_READ, 3, 0, 4096)); console.log("rax=1 ->", trap(NR_WRITE, 1, 0, 12)); console.log("rax=39 ->", trap(NR_GETPID)); console.log("rax=999->", trap(999)); // bogus number: rejected console.log("rax=60 ->", trap(NR_EXIT, 0));

What a crossing costs

A regular function call is a handful of cycles. A round trip through the kernel is hundreds — historically 150 to 500{+} cycles on x86, and made dramatically worse after 2018 by the Spectre/Meltdown mitigations, which force page-table isolation (a full address-space switch) and speculation barriers on every entry and exit. Where does the money go?

the privilege and stack switch itself (the \texttt{syscall}/\texttt{sysret} microcode);
saving and restoring the user register file;
polluted caches, TLB, and branch predictors — the kernel's working set evicts yours, so your code runs slower for a while after you return;
with KPTI, a \texttt{CR3} reload flushing much of the TLB on the way in and out.

The lesson every high-performance system learns: make fewer crossings. Three families of tricks recur throughout this course:

Serve it without a trap. The \texttt{vDSO} is a page of kernel code mapped read-only into every process; calls like \texttt{gettimeofday} and \texttt{clock\_gettime} read a shared kernel-maintained timestamp in user mode — no boundary crossing at all.
Batch many operations per crossing. \texttt{readv}/\texttt{writev} do scatter-gather in one call; \texttt{io\_uring} lets the program push hundreds of I/O requests into a shared ring buffer and reap the completions with, in the limit, zero syscalls.
Cache the answer in user space. Buffered I/O (\texttt{stdio}) turns a thousand one-byte \texttt{write}s into one big \texttt{write}.

The original 386 entered the kernel via \texttt{int 0x80}, a general interrupt gate — and general is slow, because the CPU must read the descriptor from the IDT, do privilege checks, and push a full interrupt frame. As clock speeds rose in the late 1990s the fixed overhead of a gate became a real tax on syscall-heavy workloads. Intel's answer (and AMD's, for 64-bit) was a pair of purpose-built, no-frills instructions — \texttt{sysenter}/\texttt{sysexit} and later \texttt{syscall}/\texttt{sysret} — that skip the descriptor lookup by keeping the kernel entry point and stack in fast MSRs. It cut the raw crossing from a few hundred cycles to well under a hundred. Then Meltdown arrived in 2018 and page-table isolation quietly gave much of that speed back — a reminder that the syscall boundary is where security and performance fight hardest.

A frequent confusion is to equate the C function \texttt{read()} with the kernel's \texttt{sys\_read}. They are not the same thing. The libc \texttt{read()} is ordinary user-mode code that marshals your arguments into \texttt{rdi}/\texttt{rsi}/\texttt{rdx}, puts 0 in \texttt{rax}, executes the one \texttt{syscall} instruction, and then unpacks the result. You can prove it to yourself by running \texttt{strace}: it shows the syscalls, and a single \texttt{printf} may perform zero syscalls (buffered) or one \texttt{write} — never a "printf syscall", because there is no such thing. Keep the two layers distinct: the library gives you a comfortable C API; the ABI is the register contract underneath.