Clock Synchronization and NTP

Two servers in the same rack, bought on the same day, running the same software, will disagree about what time it is — and the gap grows every hour you leave them alone. It is not a bug; it is physics. A computer's clock is a tiny quartz crystal oscillating at a nominal frequency, and no two crystals oscillate at exactly the same rate. One machine's "one second" is fractionally longer than another's, so their readings drift apart, silently, forever.

In a single computer this rarely matters. In a distributed system — where a transaction touches three machines, a log stitches events from ten, and a certificate's validity hinges on "now" — disagreeing clocks cause real, expensive bugs: a file appears to be modified before it was created, a cache entry expires in the past, distributed logs interleave nonsensically. This page is about why clocks drift, the clever algorithms that drag them back into rough agreement (Cristian's, Berkeley, and NTP), and the crucial limit lurking underneath: even perfectly synchronised clocks cannot reliably order events across machines.

Skew and drift: two different ways clocks disagree

Precision here demands two distinct words that beginners blur together.

Clock skew — the instantaneous difference between two clocks' readings at one moment. "Right now, server A reads 12:00:03 and server B reads 12:00:05" → a skew of 2 seconds. It is a snapshot.
Clock drift — the rate at which that difference grows, because the clocks tick at slightly different frequencies. It is a slope: e.g. "50 parts per million" means the clock gains or loses 50 microseconds per second, roughly 4 seconds per day.

The relationship is exactly like position and velocity: skew is where the clocks are now; drift is how fast the skew is changing. You can zero the skew by setting both clocks equal — but if they drift, the skew reopens immediately. That is why synchronisation is never "set it once": it must repeat forever, nudging clocks back before drift pulls them apart again. A typical quartz clock drifts tens of ppm; left unsynchronised for a month it can be off by minutes.

Cristian's algorithm: ask a time server, correct for the trip

The obvious fix: ask a machine that has the accurate time (say, one with a GPS or atomic clock) "what time is it?" and set your clock to the reply. The catch is the network delay: by the time the answer reaches you, it is already stale by however long the trip took. Cristian's algorithm (1989) corrects for this with a beautifully simple estimate.

The client records its send time T_0 and fires off a request.
The server, on receiving it, replies with its current time T_s.
The client records the reply's arrival time T_1.

The whole round trip took \text{RTT} = T_1 - T_0. If we assume the request and reply each took half the round trip, then the server's clock, by the time the reply lands, reads about T_s + \text{RTT}/2. So the client sets its clock to:

T_{\text{new}} = T_s + \frac{T_1 - T_0}{2}

The error is bounded by half the round-trip time (the trip could have been lopsided), so the shorter and more symmetric the path, the better the estimate. This "measure the round trip, assume it split evenly" idea is the seed of every time-sync protocol, including NTP.

The Berkeley algorithm: average instead of trusting one oracle

Cristian's assumes there is an authoritative clock. The Berkeley algorithm (1989) takes the opposite stance: no machine's clock is trusted more than any other's; agree on the average. It is used when you just want a set of machines to agree with each other, even if none of them has the "true" time.

One machine is elected the coordinator (master).
It polls every other machine for its clock reading and measures the round-trip to each (so it can correct for network delay, Cristian-style).
It computes the average of everyone's (delay-corrected) clocks — often discarding outliers so one wildly wrong clock can't skew the result.
Instead of broadcasting the new time, it sends each machine the adjustment it should apply ("speed up by 12 ms", "slow down by 30 ms").

Sending adjustments rather than absolute times matters: it lets clocks be slewed gradually (nudged faster or slower) rather than jumped, which avoids a clock ever going backwards — a jump backward can make a later event appear to happen before an earlier one, breaking any code that assumes time moves forward. Berkeley is internal synchronisation (agree with each other); Cristian's and NTP are external (agree with a reference).

NTP: the Internet's clock, in a stratum hierarchy

The Network Time Protocol (NTP) is Cristian's idea, industrialised and running on essentially every connected device on Earth. Its two big ideas are a hierarchy of sources and a four-timestamp exchange that separates offset from delay.

Time flows down a tree of strata:

Stratum 0 — the reference clocks themselves: atomic clocks, GPS receivers. Not on the network directly.
Stratum 1 — servers directly attached to a stratum-0 source; the primary time servers.
Stratum 2 — servers that sync from stratum-1 servers over the network.
Stratum 3, 4, … — each level syncs from the one above; your laptop is typically stratum 3 or 4.

A higher stratum number means "further from the true source," and NTP prefers lower-stratum, lower-delay peers. The hierarchy keeps the whole Internet's clocks anchored to a handful of atomic references without everyone hammering the same few servers.

The four timestamps are collected on one request/reply:

t_0 — client's send time (client clock)
t_1 — server's receive time (server clock)
t_2 — server's send time (server clock)
t_3 — client's receive time (client clock)

From these, NTP computes two quantities. The round-trip delay subtracts out the time the server spent processing (t_2 - t_1) from the total elapsed on the client:

\delta = (t_3 - t_0) - (t_2 - t_1)

And the clock offset — how far ahead (or behind) the client is relative to the server — is the average of the two apparent one-way differences:

\theta = \frac{(t_1 - t_0) + (t_2 - t_3)}{2}

The genius is the averaging: if the network delay is symmetric, the errors on the "up" trip and the "down" trip cancel, leaving a clean estimate of the true offset regardless of how long the trip took. The client then slews its clock by \theta. Let's compute both from real numbers.

// NTP offset and round-trip delay from the four timestamps. // t0,t3 are on the CLIENT clock; t1,t2 are on the SERVER clock. function ntp(t0: number, t1: number, t2: number, t3: number) { const offset = ((t1 - t0) + (t2 - t3)) / 2; // client is this far off vs server const delay = (t3 - t0) - (t2 - t1); // round-trip network time return { offset, delay }; } // Example: server clock is genuinely 100 ms AHEAD of the client. // Say the true one-way network delay is 20 ms each way. // client sends at t0 = 1000 (client time) // 20 ms later the server receives it; server clock = 1000 + 20 + 100 = 1120 // server replies "immediately"; t2 = 1120 // 20 ms later client receives; t3 = 1000 + 20 + 20 = 1040 (client time) const { offset, delay } = ntp(1000, 1120, 1120, 1040); console.log(`offset = ${offset} ms (positive => client is BEHIND the server)`); console.log(`delay = ${delay} ms (round-trip network time)`); console.log(`=> client should add ${offset} ms to its clock`); // Recovered: offset = +100 ms and delay = 40 ms, exactly as constructed, // even though we never knew the one-way delays directly. The averaging cancels them.

NTP repeats this exchange with several peers, filters out samples with high delay (they carry more error), and disciplines the local clock continuously — steering its frequency to fight drift, not just correcting the offset once. Over the public Internet it typically keeps machines within a few milliseconds of true time; on a local network, well under a millisecond.

The hard limit: synced clocks still can't order events

Here is the twist that surprises everyone. Suppose you spend all that effort and get every machine's clock agreeing to within, say, one millisecond. Can you now stamp each event with the wall-clock time and sort them to recover the true order they happened in? No — and this is fundamental, not a tuning problem.

The reason: synchronisation leaves a residual uncertainty (the offset estimate is never exact, and delay is never perfectly symmetric). If two events on two machines happen closer together in time than that uncertainty, their timestamps cannot tell you which truly came first. Worse, an event on machine A that caused an event on machine B could carry a later wall-clock timestamp than the effect, purely because A's clock was running a hair fast. You would conclude the effect preceded its cause.

The classic, costly mistake: "our clocks are synced by NTP, so I'll just compare timestamps to decide which write happened first." Do not do this. NTP gives you clocks that agree approximately, not clocks that are identical. There is always a residual skew of milliseconds (sometimes much more after a network hiccup), and any two events that fall inside that window can be ordered backwards by their timestamps — including a cause appearing to follow its effect.

Real systems have lost data this way: a "last-write-wins" conflict resolver that trusts wall-clock timestamps will silently discard the newer write whenever the writer's clock happened to run slow. To order events by causality — "did X happen before Y in a way that could matter?" — you need logical clocks (Lamport timestamps, vector clocks), which track the happens-before relation directly and never lie about causal order, regardless of how badly the physical clocks are behaving. Physical sync is for human-facing timestamps and timeouts; logical clocks are for ordering.

Google's Spanner database confronts the ordering problem head-on with a service called TrueTime. Instead of pretending its clocks are exact, TrueTime embraces the uncertainty: every time query returns an interval [t_{\text{earliest}}, t_{\text{latest}}] that is guaranteed to contain the true time, kept tight (single-digit milliseconds) using GPS and atomic clocks in every data centre.

The trick is what Spanner does with that interval: to commit a transaction at time t, it deliberately waits out the uncertainty — it pauses until it is certain that t has passed on every clock (until t < t_{\text{earliest}} everywhere). By waiting a few milliseconds it buys a genuine global ordering guarantee. It is a striking lesson: you can't make clocks perfect, but if you can bound how wrong they are, you can wait long enough to be safe. Most systems, which can't afford atomic clocks everywhere, fall back on logical clocks instead.