Thread pools: `ExecutorService`, `Callable`, `Future`

Threads are not free, and there is a ceiling

One thread per task is fine while tasks are few. Creating and reaping a thread costs real work in the VM, each live thread holds a stack, and once enough threads exist to keep the machine's idle time occupied, adding more only buys context switches. Spawn one per line of a 200,000-line log and you have written a denial-of-service attack against your own machine.

The fix is to decouple how many tasks there are from how many threads there are. A thread pool creates a fixed set of worker threads once, feeds them from a queue, and reuses each one across many tasks. Executors.newFixedThreadPool(n) hands you an ExecutorService with exactly n workers and an unbounded queue in front of them.

submit, and what comes back

You submit work rather than starting threads:

• submit(Runnable) returns a Future<?> whose get() yields null — you only want it to know the task finished.
• submit(Callable<T>) returns a Future<T> carrying the task's return value.

Callable<T> is Runnable's useful sibling: its single method call() returns a value and may throw a checked exception. Both restrictions on Runnable.run() disappear, which is precisely what a network task needs — a hostname to return and an UnknownHostException to raise.

submit returns immediately. The Future is a receipt: isDone() asks without waiting, cancel(boolean) tries to withdraw the task, and get() collects the answer.

get() is where the blocking moved to

get() blocks until the task finishes, then returns its value. If the task threw, get() throws ExecutionException with the original exception available from getCause() — the failure crosses the thread boundary wrapped, not lost. get(timeout, unit) adds a TimeoutException so a wedged task cannot hang the collector forever.

So the blocking never disappeared; it moved to a place you choose. Choose it late. Future<X> f = pool.submit(task); X x = f.get(); on consecutive lines is a very expensive way to call a method: you submit one task and immediately stall until it is done, so the pool has exactly one task in flight and the other workers idle. Submit everything first, then collect.

Shutting down

Pool threads are non-daemon, so a JVM whose main() has returned will sit there forever with a live pool. Ending the program is your job:

• shutdown() — refuse new tasks, run everything already queued, then stop. The orderly one.
• shutdownNow() — also skip the queue and interrupt what is running; it returns the tasks that never started.
• awaitTermination(timeout, unit) — block until the pool is finished, returning false on timeout.

Worked example — sizing the pool for DNS

You have 5,000 addresses to resolve on a four-core machine. newFixedThreadPool(4) is the wrong answer: the tasks are almost pure waiting, so four workers leave three cores' worth of wait unexploited. For I/O-bound work a useful starting point is cores × (1 + wait ÷ compute); with a 40 ms lookup and microseconds of parsing, that ratio is enormous and the real limit is elsewhere — the resolver library, the local DNS server, and the file descriptors you are willing to spend. Tens of threads is the right order of magnitude, thousands is not, and the only way to pick the number is to measure it with the log you actually have. Make it a named constant, not a literal buried in main().