Why non-blocking I/O — and the case against it

The gap that starts the argument

Line up the numbers a server actually lives with. A modern CPU moves data to and from main memory at gigabytes per second. It reaches a solid-state disk at hundreds of megabytes per second. A fast local network tops out around a hundred megabytes per second, and a consumer internet link delivers single-digit megabytes. Between memory and the wire sit two to three orders of magnitude, and nothing on the horizon closes the gap — CPUs, disks and networks all get faster together.

So a thread that calls read() on a socket is, statistically, asleep. It is parked in the kernel waiting for bytes that are still crossing a continent. The design question for a server is therefore never "how do I make the network faster". It is "what does the machine do while the network takes its time".

The classic answer, and what it costs

The traditional Java answer is blocking I/O plus threads: one thread per connection, each blocking on its own socket, the operating system running whichever threads have data. It composes with everything you already know — try-with-resources, BufferedReader, an ExecutorService — which is exactly why Modules 6 through 8 could be written the way they were.

The cost is per connection. Each thread reserves its own stack (on a 64-bit VM, on the order of a megabyte of address space by default), occupies a kernel scheduling entity, and costs a context switch every time the scheduler moves off it. At fifty connections that is invisible. At fifty thousand it is the entire budget.

Non-blocking I/O attacks precisely that cost. One thread asks the operating system "which of these ten thousand sockets can I act on right now?", services the ones that answer, and asks again. A connection stops costing a thread and starts costing a few hundred bytes of bookkeeping.

The honest counter-argument

Here is the part most tutorials leave out. Carefully architected non-blocking I/O did dramatically outperform thread-based designs — in the 1990s. Java did not get the API until version 1.4 in 2002, and by then operating systems had made native threading cheap: almost all of the context-switch and uncontested-locking overhead that justified the technique had already been engineered away, server memory had grown until ten thousand threads fit comfortably, and multicore hardware actively wanted more threads rather than fewer. Measured on Java 6 under Linux, multithreaded classic I/O beat NIO by roughly 30%.

So NIO is not "the fast one". It is a different resource trade, and on most workloads it is the losing side of that trade: you pay in complexity — a hand-rolled state machine per connection, partial reads and partial writes everywhere, one bug that stalls every client instead of one — to save a resource you were never short of.

Where a selector genuinely wins

One profile still favours it: very many long-lived connections that are individually almost idle. Ten thousand or more sockets, each sending a little data occasionally. Picture a head-office server collecting transactions from every till in a national retail chain. Thread-per-connection spends gigabytes of stack on threads that are asleep 99.99% of the time; a selector services all of them from a handful of threads.

And the two rules of optimisation apply at full strength. First: do not. Second, for experts only: do it once you hold measurements that both prove the problem is real and will tell you whether your change fixed it.

Worked example — pick the model for three servers

A departmental file server. Up to 50 simultaneous downloads, each streaming hard. Every thread is doing real work almost all of the time, so no thread is being wasted. Thread-per-connection, and stop thinking about it.

A public web front end. Around 500 short requests in flight at once. Unbounded thread creation is a self-inflicted denial of service, so use the fixed ExecutorService pool from Module 7: bounded resource use, and every handler stays plain blocking code that a human can read.

A telemetry collector. 20,000 sensors, each sending a 40-byte reading once a minute. That is about 13 kB/s of real traffic. Thread-per-connection would reserve on the order of 20 GB of stack to carry it, and every one of those threads would be blocked in read() essentially forever. This is the workload that pays for a selector — and it is the only one in the list that does.