Thread-pooled servers

A pool is a ceiling you choose

Thread-per-connection fixed the latency problem and created a resource problem: the thread count is set by whoever is connecting. A thread pool takes that decision back. Executors.newFixedThreadPool(50) returns an ExecutorService holding at most fifty worker threads and an internal queue of pending tasks. You submit() a task; if a worker is free it runs at once, and if all fifty are busy it waits in the queue until one finishes. Threads are created once and reused for the life of the server, so the per-connection cost drops to enqueueing an object.

The pool size is therefore the maximum number of clients being served simultaneously, chosen by you at compile time and unaffected by load. That is the whole point. Under a flood, a pooled server degrades — clients wait, and eventually the kernel queue fills and connections are refused — but it does not die, and the clients already inside are still served at full speed. Graceful degradation under overload is what back-pressure means, and it is the difference between a server that has a bad afternoon and one that has to be restarted.

Runnable, Callable, and where the exception goes

submit() accepts either a Runnable or a Callable<V>, and returns a Future<V> either way. The Callable form is worth knowing because call() may throw checked exceptions, which Runnable.run() may not — handy when a handler wants to let an IOException out.

But there is a sharp edge. An exception thrown out of a submitted task does not print. It is captured inside the Future and stays there until somebody calls Future.get() — and in a server nobody ever does, because the return value is meaningless. So a NullPointerException in your handler vanishes without a trace, the client gets a truncated response, and your console stays clean. Catch inside the task, log there, and treat the Future as a formality.

Sizing an I/O-bound pool

The familiar advice — one thread per core — is for CPU-bound work, where extra threads only add context switches. A connection handler is I/O-bound: it spends nearly all its life blocked on a socket read or a disk read, consuming no CPU at all. Its pool can therefore be many times the core count, and the real limits are memory (a stack per thread) and file descriptors (a socket per client). Fifty is a sound default for a small file server on a laptop; the honest way to choose is to measure how long a handler blocks versus how long it computes, and to be able to say why your number is your number.

Shutting down in the right order

Order matters, because the accepting thread is blocked in accept() and cannot see a flag.

1. listener.close() — this is what wakes the blocked accept(), which throws a SocketException. That exception is the loop's exit signal, not an error.
2. pool.shutdown() — refuses new tasks, lets the queued and running ones finish.
3. pool.awaitTermination(10, TimeUnit.SECONDS) — block until they do, or until you run out of patience.
4. pool.shutdownNow() if that returned false — interrupt whatever is still running.

Skip step 2 and the JVM will not exit: pool threads are non-daemon by default and keep the process alive forever.

Worked example — the pooled daytime server

public final class PooledDaytimeServer {

    private static final int POOL_SIZE = 50;

    public static void main(String[] args) throws IOException, InterruptedException {
        ExecutorService pool = Executors.newFixedThreadPool(POOL_SIZE);
        ServerSocket listener = new ServerSocket(1313);
        Runtime.getRuntime().addShutdownHook(new Thread(() -> quietClose(listener)));

        try {
            while (true) {
                pool.submit(new DaytimeHandler(listener.accept()));
            }
        } catch (SocketException ex) {
            // the shutdown hook closed the listener; fall through and drain
        } finally {
            pool.shutdown();
            if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
                pool.shutdownNow();
            }
        }
    }

    private static void quietClose(ServerSocket listener) {
        try {
            listener.close();   // wakes the blocked accept() with a SocketException
        } catch (IOException ignored) {
            // nothing useful to do while the JVM is exiting
        }
    }
}

DaytimeHandler is the same class as the last lesson's — a Runnable holding one Socket and closing it in run(). That is the pattern to keep for Assignment 2: the accept loop never changes, only the handler does.