Crash Consistency and fsck

Here is the problem that haunts every file system and drives the next three lessons. You know that a single file operation — say, appending one block to a file — is logically one action. But on disk it is several separate writes: the new data block itself, the inode (updated size and a new block pointer), and the free-space bitmap (that block flipped from free to used). Three writes, and the disk does them one at a time, in whatever order it likes.

Now pull the power cord in the middle. Some of those three writes made it to the platter; some did not. The file system reboots into a state that no correct sequence of operations could ever have produced — a bitmap that disagrees with the inodes, an inode pointing at a block marked free, a used block that no file owns. This is the crash-consistency problem, and it is fundamental: it exists because a disk gives you single-block atomicity but a file operation needs multi-block atomicity, and the two do not match. Everything that follows — journaling, log-structured, copy-on-write — is an answer to this one question.

The canonical example: appending a block

Take a file with one block, and append a second. The file system must perform three writes:

There are 2^3 = 8 possible outcomes for which subset of writes survives. Two are fine (none landed → as if nothing happened; all landed → success). The other six are trouble, and they are trouble in different ways — some merely leak space, some hand out a block that is still in use, some leave the inode pointing at garbage. This variety is exactly why the problem is subtle.

What landedResulting state
only Iinode points at a block the bitmap calls free → will be handed to another file → corruption
only Bblock marked used but no inode owns it → space leak (lost forever)
only Dbbytes on disk but nothing references them → harmless, but the append is lost
I + Db, no Binode and data agree, but bitmap says free → block can be double-allocated → corruption
I + B, no Dbmetadata consistent, but the data block holds garbage (stale bytes)
B + Db, no Iblock used and written, but no inode points to it → space leak

See a crash land a prefix

Lay the three writes on a timeline. A crash draws a vertical line somewhere along it: everything to the left made it to disk, everything to the right vanished. Depending on where the line falls — and the order the disk chose to do the writes — you land in one of the eight states above.

Enumerate the damage

The model below runs through all eight subsets of {B, I, Db} and classifies each outcome — consistent, a space leak, or outright corruption. Read the output and you have the whole crash-consistency problem in front of you: most partial outcomes are broken, and they are broken in incompatible ways, so there is no single simple patch.

// Enumerate every subset of the three writes an append needs, and classify the crash outcome. // B = bitmap marks the block used, I = inode points at it, Db = data bytes are written. type W = { B: boolean; I: boolean; Db: boolean }; function classify(w: W): string { if (!w.B && !w.I && !w.Db) return "consistent (nothing happened — append simply lost)"; if (w.B && w.I && w.Db) return "consistent (full success)"; // Metadata cross-checks: if (w.I && !w.B) return "CORRUPTION (inode points to a block the bitmap calls free)"; if (w.B && !w.I) return "SPACE LEAK (block marked used but owned by no inode)"; if (w.I && w.B && !w.Db) return "consistent-metadata but GARBAGE data in the block"; if (w.Db && !w.I && !w.B) return "consistent (orphan bytes, referenced by nobody)"; return "unclassified"; } for (const B of [false, true]) for (const I of [false, true]) for (const Db of [false, true]) { const tag = `${B ? "B" : "-"}${I ? "I" : "-"}${Db ? "d" : "-"}`; console.log(`landed ${tag} : ${classify({ B, I, Db })}`); }

fsck: the scan-and-repair sledgehammer

The oldest answer is not to prevent inconsistency but to detect and repair it after the fact. fsck (file-system check) runs at boot after an unclean shutdown and walks the entire file system, rebuilding the metadata to a self-consistent state:

It works — but it has two brutal limitations. First, it is O(size of the file system): it must read every inode and often every block, so on a multi-terabyte disk a full \texttt{fsck} can take hours, during which the machine is down. As disks grew from megabytes to terabytes, this went from "a few seconds at boot" to "unacceptable," and it is the direct historical reason journaling was invented.

Second, and worse: fsck restores consistency, not correctness. It can make the bitmap agree with the inodes, but if the crash left your inode pointing at a block that was never written, fsck cannot conjure the data you meant to save — it can only make the file system coherent about the fact that the data is garbage. Your last transaction is simply gone. fsck saves the file system; it cannot save your file.

Because "everything" is scattered across the disk — the inode lives in the inode table, the bitmap in the allocation area, the data block out in the data region — and a disk can only guarantee that a single sector (or block) is written all-or-nothing. There is no hardware primitive that says "write these three far-apart blocks together or not at all." You could try to make them adjacent, but a file's inode, its bitmap, and its data fundamentally live in different structures. The entire field of crash consistency exists to build multi-block atomicity out of the single-block atomicity the hardware gives you — by logging intentions first (journaling), by never overwriting (copy-on-write), or by turning all writes into one sequential stream (log-structured). Each is a different bridge across the same gap.

The single most dangerous misreading of this topic is to treat crash consistency as a durability guarantee. They are different promises. A crash-consistent file system guarantees only that after recovery its structures are coherent — no dangling pointers, no double-allocated blocks — so it will not corrupt other files or crash the kernel. It says nothing about whether your just-written data survived; the whole transaction may have vanished, and after fsck the file system is perfectly happy about that. If you need "this data is definitely on stable media before I continue," that is a durability request — you must call \texttt{fsync}, the subject of a later lesson. Confusing the two is behind a long list of real-world data-loss bugs.

Where this leads

fsck's two flaws — it is O(disk) slow, and it cannot recover lost updates — motivate everything that comes next. Journaling writes intentions to a small log first, so recovery replays only the log instead of scanning the whole disk. Log-structured and copy-on-write file systems go further and make inconsistency structurally impossible. Keep the eight-outcome picture in mind — each design is a different way of collapsing those six bad states.