Grounded

Wednesday’s fix does not touch the model at all, which is the whole cleverness of it.

The pasted excerpt was the flaw. A static page of greatest hits can never cover deferral questions, refund edge cases, or whatever 25,000 people ask next, and everything outside the excerpt invited likelihood to improvise. The replacement: when a question arrives, search Harborview’s entire FAQ and policy corpus for the passages most relevant to it, paste those into the prompt, and instruct the model to answer only from them. The pattern is retrieval-augmented generationConcept · lights on your mapretrieval-augmented generationRAG: when a question arrives, search your own data for the most relevant pieces, paste them into the prompt as context, and instruct the model to answer from them. The model stays unchanged; you hand it the right documents at the right moment. The dominant pattern for AI-with-your-data, and it kills most hallucination by giving truth to be likely with., and it attacks Tuesday’s failure at the root. The model invents when likelihood has nothing true to lean on. Put the actual deferral policy, which says no such thing exists, in the window, and the likeliest continuation becomes the truth.

Finding “keep my spot”

The subtle part is the searching. A runner asks “can I keep my spot if I sit out a year,” the policy is titled “entry deferral eligibility,” and no keyword joins them. The search works on embeddingsRecognition — just know it existsembeddingsNumeric representations of text where similar meanings land near each other, which is how retrieval finds “keep my spot” in a policy that says “deferral eligibility.” Meaning-based search, not keyword matching. Recognition tier: nod when the AI engineers say it., meaning mapped to numbers so that neighbouring ideas sit as neighbouring points, and recognition depth is all you need. The engineers say embeddings, the search finds meaning, the right paragraph lands in the window.

Dana asks the good product question: why not just teach the model Harborview’s policies permanently? That option exists, and it is fine-tuningConcept · lights on your mapfine-tuningAdditional training on your own examples, actually adjusting the model’s weights, to teach persistent style, format, or domain behavior. Slower and costlier than RAG, and wrong for fast-changing facts. The clean split: fine-tuning for how to behave, RAG for what is true right now., and it answers a different need. Weights are for behavior: tone, format, the house way of declining. Facts that change with every race season belong in retrieval, where updating the truth is editing a document instead of retraining a model. Traversal wants Wednesday’s policy answered with Wednesday’s policy. RAG it is.

By evening the assistant, asked about deferrals, quotes the real policy and apologizes for the confusion. The eval suite runs overnight. Invention on uncovered questions: measurably, satisfyingly down. What has not changed is the model itself, still rented, still remote, still somebody else’s machine, and that arrangement is Thursday’s last lesson.