The trace

At 12:34 by the ops-room clocks, Case crosses the line: four hours, thirty-one minutes and seven seconds by the chip’s own counting, start mat to finish. Case stops running, and stands in the particular stillness of somebody who has just spent four months keeping a promise. Four hours, thirty-one minutes, seven seconds. Eleven minutes and change slower than the kitchen-table guess, exactly as foretold.

In the ops room, the afternoon’s work has changed key: from watching curves to answering questions. The race director wants assurance the morning’s turbulence touched no result, and assurance, in this room, is a discipline rather than a feeling: debuggingConcept · lights on your mapdebuggingThe systematic hunt for why software misbehaves: reproduce, narrow, verify, powered by evidence and by the eternal first question, what changed., the evidence-first hunt, pointed for once at proving a negative. Did the degraded morning lose anything?

One runner, end to end

Finn picks a runner at random, then picks a runner not at random, and pulls the traceConcept · lights on your maptraceThe linked record of one request’s journey across every service it touched, with timing at each hop. A log tells one service’s side; the trace tells the whole conversation. for bib 11208: the linked record of one crossing’s journey through every box it touched. Mat six, 11:06:02. Ingest, two milliseconds later. Queue, worker, results write, the fan-out to three consumers, Molly’s buzz: every hop stamped, every stamp inside its budget, straight through the worst twenty minutes of the morning. Logs are one service’s diary. The trace is the whole conversation, and reading one is the closest engineering gets to replaying time.

Somewhere in the afternoon’s spelunking Finn also finds an actual bugConcept · lights on your mapbugThe code-level flaw itself, met here at its most consequential: the thing an incident investigation hunts. Not every incident traces to one; configuration, capacity and dependencies star regularly too., a small one, in the cheer feature of all places: messages sent during the degraded window carry a wrong timestamp, cosmetic, harmless, noted for Monday. Worth saying plainly: the morning’s incident was not a bug. The code did what it was built to do. The world supplied a bridge, a crowd, and thin air, and capacity met weather. Incidents trace back to something — only sometimes is the something a mistake.

The results export leaves for the race director at 5:44, sixteen minutes inside the SLA, a CSV exactly as promised in Act I. Case, medal on, eats the largest meal of this entire story. Tomorrow, the telling of the truth about the morning. And tomorrow also brings an email the afternoon’s clean trace makes genuinely strange.