The second mystery

Monday morning, one week and a day after the gun, Dana walks in holding a printout of the week’s strangest complaint. A runner who finished in just over five hours reports that, somewhere past halfway, the app told their family to expect them at 3:58.

The projected finish. It updated within the half-second, it was usually eerily right, and for one runner on one Sunday it was spectacularly wrong, which makes it this act’s second mystery. Solving it requires opening a box this story has walked past since Act II, the one on the whiteboard labelled artificial intelligenceConcept · lights on your mapartificial intelligenceThe broad field of making machines do things that seem to require intelligence. In products today the word overwhelmingly means one family: machine-learning models, and especially large language models..

Software nobody wrote

Every box so far held instructions somebody wrote. This one is different in kind: a machine-learning modelConcept · lights on your mapmachine-learning modelSoftware whose behavior was learned from data rather than written as rules: show it millions of examples, and training adjusts millions of internal numbers until useful patterns emerge. Nobody writes, and nobody can fully read, the resulting behavior. Less a recipe, more a trained intuition.. No engineer wrote a rule that says fading runners fade further. The model absorbed ten years of Harborview history, every split by every finisher for a decade, and whatever regularities live in that record now live, unreadably, in its numbers.

Learning and using are two different lives. trainingConcept · lights on your maptrainingLearning from data: adjusting a model’s internal numbers against examples until patterns hold. At the frontier it is months long, monumentally expensive, and done rarely, by AI labs on warehouses of GPUs. Almost no product team trains at that scale; what products do, constantly, is inference. happened before race week: the data team’s overnight runs, chewing the decade. What happened 240 milliseconds after Case’s crossings was inferenceConcept · lights on your mapinferenceUsing an already-trained model to answer one question: the per-request act a product actually performs, often by calling a provider’s API. The cheap, frequent half. Training is the rare, expensive one., the trained intuition consulted, per runner, per mat, all morning. At any serious scale, both lives lean on the same peculiar hardware: the GPUConcept · lights on your mapGPUThe graphics processor repurposed as AI’s workhorse: thousands of simple cores doing arithmetic in parallel, exactly what the matrix math of models demands. Scarce, expensive, and the reason “compute” became a boardroom word.. Act II’s CPU, a brilliant soloist, has a sibling that is an orchestra of simple players, and model math wants the orchestra.

The one that talks

One more distinction before the mystery can crack. The projection model predicts a single number and could not write a sentence to save its life. The famous kind, the large language modelConcept · lights on your maplarge language modelA model trained on vast amounts of text to predict what comes next, which at sufficient scale yields answering, summarizing, translating, and writing code. The engine of every AI assistant, and “LLM” in every meeting., is a different animal: trained on text, predicting the next fragment of a sentence. Calling the projection an LLM would be false, and this story will not do it. But Traversal does have one of those. It answers runners’ questions in the help widget, it talks all day, and this evening it earns its own explanation. The 3:58 and the talker, it turns out, fail in instructively different ways.