Worked build: a pooled weblog analyser

The record you are parsing

Web servers log hits in the common log format, one line per request, seven fields:

198.51.100.7 - alice [03/Aug/2026:09:14:22 -0600] "GET /notes/index.html HTTP/1.1" 200 4213

That is: the client address, the (essentially dead) rfc931 identity field, the authenticated user or -, a bracketed timestamp, the quoted request line, the status code, and the number of bytes sent. Two fields contain spaces, so you cannot blindly split the line into seven pieces — but you do not need to. Everything this program wants is at the ends: the address is everything before the first space, and the byte count is the last token. A - in the byte field means no body was sent (a 304, say) and Long.parseLong will throw on it.

Servers can be configured to log hostnames instead of addresses, and you should never turn that on: it makes the server perform a DNS lookup on the hot path of every hit. Log the address, resolve later, off the box — which is exactly the program you are about to write.

Stage 1 — serial, and why it is unusable

The naive version is fifteen lines: read a line, cut the address off the front, call InetAddress.getByName(address).getHostName(), print the name where the address was. It is correct, and on a real log it is useless. Every uncached lookup is a network round trip of tens of milliseconds while the CPU does nothing, so throughput lands near one line per lookup. A week of traffic from a small site is comfortably 200,000 lines — hours of pure waiting.

The one thing the serial version has going for it is caching: InetAddress caches successful lookups, and a real log revisits the same few thousand addresses constantly (one page view produces a log line per image). That helps every version equally; it does not make this one fast enough.

Stage 2 — put the whole per-line job on the pool

The work for one line is a single blocking call wrapped in a little string surgery, which is exactly the shape a pool is for. The design decision worth thinking about is what the task hands back. A rewritten log line is the obvious answer and the wrong one — the collector would only have to take it apart again to count anything. Hand back the two values the report is actually made of:

record Hit(String host, long bytes) { }

final class ResolveHit implements Callable<Hit> {

    private final String entry;

    ResolveHit(String entry) { this.entry = entry; }

    @Override public Hit call() {
        int firstGap = entry.indexOf(' ');
        int lastGap = entry.lastIndexOf(' ');
        if (firstGap < 1 || lastGap <= firstGap) {
            throw new IllegalArgumentException("unparseable record: " + entry);
        }
        String sent = entry.substring(lastGap + 1);
        long size = "-".equals(sent) ? 0L : Long.parseLong(sent);
        return new Hit(nameFor(entry.substring(0, firstGap)), size);
    }

    private static String nameFor(String address) {
        try {
            return InetAddress.getByName(address).getHostName();
        } catch (UnknownHostException noReverseRecord) {
            return address;          // keep the hit, key it by the literal address
        }
    }
}

Three choices to defend in your documentation. The task parses as well as resolves, so the DNS wait and the string work land on the same pooled thread and the collector receives finished values it never has to re-split. An address with no reverse record keeps its numeric form rather than vanishing: reverse lookups fail constantly in the wild, and an analyser that quietly drops every hit it could not decorate under-reports exactly the traffic it exists to measure. A line that is not a log record at all is a different failure and it throws — a total you reached by ignoring input you did not understand is worse than a total you declined to produce.

A pool, not a thread per line. newFixedThreadPool(n) with n a named constant keeps DNS pressure bounded and memory flat while the main thread reads ahead as fast as the disk allows.

Stage 3 — preserving input order

A pool completes tasks in whatever order the network allows, so consuming results as they finish scrambles the run. That costs you twice: any echo of the resolved records comes out shuffled, and so does the per-host report, because a LinkedHashMap lists its keys in first-seen order and "first seen" means nothing once the input order is gone. The fix costs one data structure: as each line is submitted, push its Future onto a queue. Then walk the queue and get() each one in turn. Each get() blocks only if that record is still outstanding, while every other task keeps running — so the report is deterministic and the concurrency is untouched. Order comes from the queue, never from the pool.

The honest cost is footprint: one Future per line, alive until you drain it. For a file too large to hold that, stop reading once the queue reaches a ceiling and let the drain make room. Bounding the submissions rather than adding a second consumer thread keeps the "am I finished?" question trivially answerable — the reader is still the only thing that decides when there is no more input.

Worked example — the three tallies behind the option flag

The assignment runs as java MyPooledWeblog logname option, where option 1 counts accesses per remote host, 2 counts total bytes transmitted, and 3 counts total bytes per remote host. All three are folds over the same resolved stream, so parse once and dispatch on the option:

• Option 1 — Map<String, Long> keyed by hit.host(), merge(host, 1L, Long::sum).
• Option 2 — one long accumulator; add hit.bytes(), no map at all.
• Option 3 — the same map as option 1, merging hit.bytes() in place of the 1L.

Use a LinkedHashMap so the report comes out in first-seen order and two runs over the same log are byte-identical — a marker comparing your output to a sample cares. Do the tallying on the collecting thread as you drain the queue: it is microseconds of work per record, it needs no lock, and a shared mutable tally updated from pooled tasks would need the machinery of L3.3 for no gain. The byte field was already sanitised inside ResolveHit, so the fold is pure arithmetic; wrap each get() in a catch (ExecutionException …) so one unparseable line costs you that line rather than the run; print the report after the drain loop; and call shutdown() before you return — otherwise the program produces a perfect report and then hangs forever.