mmap, Copy-on-Write and Demand-Zero
Call \texttt{fork()} on a process using
4\,\text{GB} of memory and the child appears essentially
instantly — with its own private, full copy of all four gigabytes. Copying four gigabytes takes
far longer than \texttt{fork} takes to return, so the copy plainly did not
happen. The whole illusion rests on the same lazy principle behind
demand
paging: don't do the work until someone actually needs it. This lesson
collects the family of tricks the OS plays on the
page
table to make copies, mappings and blank memory nearly free — mmap,
copy-on-write, demand-zero, shared memory and huge pages.
mmap: the address space is a set of mappings
Underneath, a process's address space is not a flat block — it is a collection of regions
(Linux calls them VMAs, virtual memory areas), each created by a mapping. The
\texttt{mmap()} system call is how they come into being, and there are two
flavours:
- File-backed: map a file into memory. Reading the mapped bytes faults the
corresponding file blocks in on demand (from the page cache); the pages are the file. This is
how programs and shared libraries are loaded, and how databases read files without
\texttt{read()} calls.
- Anonymous: memory backed by nothing on disk (until swapped) — this is what
\texttt{malloc} uses for large allocations, and what the heap and stack are.
Its pages start as demand-zero (below).
Each mapping is also either private (writes stay local, via copy-on-write) or
shared (writes are visible to everyone mapping it, and, for a file, written back). Those
two axes — file/anonymous × private/shared — describe almost every page in the system.
Demand-zero: a single blank page for everyone
Fresh anonymous memory must read as zero (leaking a previous owner's data would be a security hole). But
eagerly zeroing a gigabyte of freshly-allocated heap the moment
\texttt{malloc} returns would be wasteful — most of it may never be touched. So
the OS uses demand-zero pages: every not-yet-written anonymous page is mapped, read-only,
to one shared physical zero page. Reads of any such page return zeros for free, from that
single frame. Only when the process writes does a fault fire, at which point the OS allocates a
real frame, zeroes just that one, and maps it writable. You pay for a blank page only if you dirty it.
- Demand-zero: unwritten anonymous pages share one read-only zero frame; the first
write faults in a private, zeroed frame;
- Copy-on-write (COW): after \texttt{fork}, parent and
child share every page read-only; the first write by either faults, copies that one page, and gives
the writer a private copy;
- both defer expensive work (zeroing, copying) to the moment of first write, and both rely on a
write-protect fault plus a per-frame reference count.
Copy-on-write: the star of the show
Copy-on-write makes \texttt{fork} cheap. Instead of duplicating the parent's
pages, the kernel makes parent and child share every page, marks all of them
read-only in both page tables, and bumps each frame's reference count to 2. As
long as both processes only read, they read the same physical frames — no copying, ever. The moment
either writes, the write-protection triggers a fault; the kernel sees the page is COW, allocates a
fresh frame, copies the contents, maps the writer's PTE to the new frame (writable), and drops
the old frame's refcount. Only the pages that are actually written ever get copied. Step through the
divergence:
Watch the refcount, because it carries the logic. When a COW fault hits a frame whose refcount is still
> 1, the writer must get a private copy. But when the refcount has already
fallen to 1 (everyone else has diverged or exited), there is no one left to
protect — the kernel can simply re-enable write on the existing frame with no copy at all. The
reference count is the whole decision procedure.
// Copy-on-write after fork(): one shared page, tracked by a reference count.
// A write to a shared (refcount>1) frame copies; a write to a private (refcount==1) frame does not.
let nextFrame = 100;
const frame = { id: 42, refcount: 1, writable: true }; // parent's page, initially private
let copies = 0;
function fork(): { parent: typeof frame; childRefsFrame: number } {
frame.refcount = 2; // now shared by parent and child
frame.writable = false; // write-protect in BOTH page tables
console.log(`fork(): share frame ${frame.id}, refcount=${frame.refcount}, marked read-only`);
return { parent: frame, childRefsFrame: frame.id };
}
function write(who: string, f: { id: number; refcount: number; writable: boolean }): number {
if (f.writable) { console.log(`${who} writes frame ${f.id}: already private, no copy`); return f.id; }
if (f.refcount > 1) {
const nf = { id: nextFrame++, refcount: 1, writable: true }; // COW: allocate + copy this page
f.refcount--; copies++;
console.log(`${who} writes shared frame ${f.id}: COW fault -> copy to frame ${nf.id}; old refcount now ${f.refcount}`);
if (f.refcount === 1) f.writable = true; // last sharer: re-enable write, no future copy needed
return nf.id;
}
f.writable = true;
console.log(`${who} writes last-sharer frame ${f.id}: re-enable write, no copy`);
return f.id;
}
fork();
write("child", frame); // first write by child -> triggers the copy
write("parent", frame); // parent is now the lone owner -> no copy
console.log(`total physical copies made: ${copies} (out of a whole shared address space)`);
Shared memory and huge pages
Not all sharing is temporary. A shared mapping (\texttt{MAP\_SHARED},
or System V / POSIX shared memory) is the deliberate opposite of COW: two processes map the
same physical frames writable, and a store by one is instantly visible to the other. It is the
fastest possible inter-process communication — no copying, no syscalls per message — and the substrate for
databases' shared buffer pools and \texttt{/dev/shm}.
Huge pages attack a different cost. A program with a multi-gigabyte working set, mapped in
4\,\text{KiB} pages, needs millions of PTEs and blows out the
TLB — every few
accesses miss and pay a page walk. Mapping the same region in 2\,\text{MiB} (or
1\,\text{GiB}) huge pages means one TLB entry covers
512\times (or 262{,}144\times) as much memory, so the
TLB's reach explodes and misses plummet. A single 2\,\text{MiB} page
also short-circuits translation by one level (the PD entry points straight at the data), so the walk is
shorter too. The price is coarser granularity: COW or eviction of a huge page moves
2\,\text{MiB} at a time, and they need contiguous physical memory, which
fragmentation can make scarce.
COW makes \texttt{fork} cheap, but the classic
\texttt{fork()}-then-\texttt{exec()} pattern still
does needless work: \texttt{fork} dutifully write-protects the parent's
entire address space and sets up all those COW mappings — and then
\texttt{exec} immediately throws the whole lot away to load a new program.
You paid to build COW machinery you never used. That is why \texttt{vfork()}
(borrow the parent's address space briefly, promise to \texttt{exec}
immediately) and, better, \texttt{posix\_spawn()} exist; and why forking a
many-gigabyte process (a big JVM, Redis taking a snapshot) can still stutter — even at one COW mapping
per page, a huge address space is a lot of page-table fiddling.
The name says it, yet it is constantly misremembered. After \texttt{fork}, no
memory has been copied — parent and child genuinely share the same physical frames, and reads
from either touch identical bytes at zero cost. Copying is triggered only by a write, and
only for the one page written, at the moment it is written. So a child that only reads a huge
inherited buffer costs nothing; a child that scribbles one byte into each page slowly copies the whole
thing. This also explains a real gotcha: a garbage collector or reference-count update that writes
to objects (even just touching a header) in a forked child will silently copy pages you thought were
shared — the reason Redis and CoW-based snapshotting care so much about not dirtying pages. "Copy-on-write"
is a promise about writes; reads are always free.