Non-Blocking and Event-Driven I/O

A single waiter can serve a hundred tables — but not by standing frozen at table 1 until its guests decide what to order. A good waiter circulates: they glance across the room, notice which tables have raised a hand, and attend only to those, then glance again. The tables that are still reading the menu cost the waiter nothing. This is, almost exactly, how one thread can serve tens of thousands of network connections — the trick the last page promised, and the engine inside nginx, Node.js and Redis.

We saw that one thread per connection hits a wall around ten thousand connections. The escape is to stop giving each connection a thread to block on, and instead let one thread ask the kernel "which of my connections are ready right now?" and service only those. That requires two ingredients: non-blocking sockets (so a read on an idle socket returns immediately instead of freezing the thread) and an I/O multiplexing system call (so the thread can wait on all its sockets at once). Put them together and you get the event loop.

Blocking vs non-blocking

By default a socket is blocking: call read() when no data has arrived and the calling thread is parked by the kernel until some does. That is fine when a thread has exactly one connection to care about — but fatal when it has thousands, because parking on connection 1 makes it deaf to the other 9,999.

Flip the socket to non-blocking and the same read() on an empty socket returns instantly with a special "nothing yet" signal (EWOULDBLOCK / EAGAIN) instead of parking. Now the thread never gets stuck — but you have created a new problem: if you just loop over all your sockets calling non-blocking read(), you busy-spin, burning 100% CPU asking "anything? anything? anything?" of sockets that are almost all idle. You need the kernel to tell you which sockets are worth touching. That is exactly what the multiplexing calls do.

select → poll → epoll: asking "who's ready?"

Three generations of system call answer the question "of these N sockets, which are ready to read or write?" — and the story of their evolution is the story of solving C10K.

select() (1983). You hand the kernel a bitmap of your sockets; it returns which are ready. Two fatal limits: it scans all N sockets on every call (O(N) work even if only one is ready), and the fixed-size bitmap caps you at FD_SETSIZE (typically 1024) descriptors.
poll() (1986). Drops the 1024 cap by taking an array instead of a bitmap. But it is still O(N): you pass the entire list of sockets on every single call, and the kernel walks all of them, every time, to find the few that are ready.
epoll() (Linux, 2002) and kqueue() (BSD/macOS). The breakthrough. You register your sockets with the kernel once; thereafter a call returns only the ready ones. The cost is O(number of ready events), not O(total sockets) — so a server monitoring 100,000 connections of which 50 are active does work proportional to 50, not 100,000. That single change is what made the C10K wall crumble.

The picture below is the whole argument. select/poll cost climbs linearly with the number of connections you monitor, because they re-scan everything each time. epoll stays flat, because its cost tracks only how many sockets are actually ready.

select/poll are O(N) in the number of monitored descriptors — you resubmit and the kernel rescans the full set on every call.
epoll/kqueue are O(number of ready events) — register once, then pay only for sockets that actually have activity. Idle connections are effectively free.

The event loop (reactor pattern)

Wrap all this in a loop and you have the reactor pattern, the beating heart of every high-concurrency server. One thread, forever:

Wait — ask the kernel (via epoll) for the set of ready sockets, sleeping efficiently if none are.
Dispatch — for each ready socket, call its registered callback (the "handler" for that connection): read the available bytes, do a little work, maybe queue a write.
Repeat.

There is no per-connection thread and no blocking: an idle connection simply never shows up in the ready set, so it costs nothing. This is precisely how Node.js (libuv), nginx, and Redis serve enormous connection counts on essentially a single thread. The demo below runs a miniature reactor: several ticks arrive, each with a different subset of sockets ready, and the loop touches only the ready ones — while a comparison counts what a poll()-style "scan everything" loop would have cost.

Readiness vs completion; and callback hell

Two refinements finish the picture. First, a subtle distinction in what the kernel tells you.

Readiness model (epoll, kqueue). The kernel says "this socket is ready — you may now read without blocking," and you perform the read. This is the classic reactor.
Completion model (io_uring on Linux, IOCP on Windows). You say "please read into this buffer and tell me when it's done." The kernel performs the whole operation and hands you the finished result. This is the proactor pattern, and it maps naturally onto async/await.

Second, a human problem. A reactor is built from callbacks — "when this socket is readable, run this function." Chain several asynchronous steps (read request → query database → read file → write response) and you get callbacks nested inside callbacks inside callbacks: the infamous "callback hell" (the rightward-drifting pyramid of doom). The industry's answer was Promises/futures, and then async/await syntax, which lets you write the code as a flat, sequential-looking sequence while it still runs on the non-blocking event loop underneath. The event loop never went away — await is just a beautiful disguise over "give this socket a callback and yield the loop."

// Same logic, two styles. The event loop is identical underneath. // "Callback hell": each async step nests inside the previous callback. readRequest(conn, (req) => { queryDb(req.id, (row) => { readFile(row.path, (body) => { writeResponse(conn, body); // ...drifting ever rightward }); }); }); // async/await: flat and sequential to read, still non-blocking to run. async function serve(conn: Conn) { const req = await readRequest(conn); const row = await queryDb(req.id); const body = await readFile(row.path); await writeResponse(conn, body); }

The event loop's superpower — one thread for everything — is also its Achilles' heel. Because there is one thread running one callback at a time, any callback that does not return promptly stalls the entire server: while it runs, no other socket can be serviced, no new connection accepted, nothing. Non-blocking I/O does not make the work free — it only makes waiting free. A callback that does a heavy CPU computation (parsing a 50 MB JSON, hashing a password with a slow KDF, a tight numeric loop) or — worst of all — makes a blocking call (a synchronous file read, sleep, a blocking DB driver) holds the loop hostage, and every one of your ten thousand connections hangs at once.

The rules follow directly: never do heavy CPU work or any blocking/synchronous call inside the event loop. Offload CPU-bound work to a worker thread/process pool, use only non-blocking (async) library calls, and break large jobs into chunks that yield back to the loop. "Non-blocking" describes the I/O, never your own code.