Grounding and shaping: RAG and fine-tuning

A general model knows a great deal about the world and nothing about your business. It has never seen your menu, your policies, or what Erik ordered yesterday.

Closing that gap has exactly two levers, and knowing which one a problem needs is a real product skill.

What is retrieval-augmented generation?

Lever one, the workhorse, is retrieval-augmented generationConcept · lights on your mapretrieval-augmented generationRAG, the dominant pattern for AI over your own data. When a question arrives, a search pulls the few stored passages that bear on it, those passages ride into the prompt, and the model is told to answer from them alone. Nothing about the model changes, and hallucination falls because the real answer is now inside the window., RAG (rhymes with bag).

Instead of hoping the model knows your refund policy, the backend fetches the policy at question-time and puts it in the window: “Here is the policy: … Now answer from it.” The model stays unchanged, an open-book test-taker handed the right pages at the right moment.

Why search for the most relevant pieces instead of pasting in everything you have? Lesson two answered that before you asked. The context window is a hard budget, and retrieval is the editor deciding what deserves it.

And hallucination drops sharply, for a reason you can now state precisely. Generation picks likely tokens, and once the actual policy is sitting in the window, the truth is the likeliest thing in sight.

Watch a RAG request assemble.

How the search finds meaning

The searching itself hides one recognition-tier term, embeddingsRecognition — just know it existsembeddingsText turned into coordinates, placed so that phrases with related meanings end up close together. Retrieval searches those coordinates, which is how a question finds a policy it shares no vocabulary with. Recognition tier: nod when the AI engineers say it.: text mapped to numbers such that similar meanings sit near each other.

That is how “can I get my money back” finds the refund policy despite sharing no words with “refund eligibility.” It is search by meaning, not by keyword. A small model does the mapping. Your documents took their coordinates ahead of time, the question takes its own on arrival, and the search returns whatever sits closest. (The numbers are stored in a database, and Module 7 extends its empire into the AI age.)

What is fine-tuning?

Lever two, the heavy one, is fine-tuningConcept · lights on your mapfine-tuningA second, smaller round of training run on examples you supply, which shifts the model’s actual weights. It buys durable style, format, or domain behavior, costs more than prompting, and is mismatched to facts that keep moving. Behavior earns tuning, and current truth rides the prompt.. Take the trained model and train it a bit more, on your examples, actually changing its internal numbers. An example is a pair, the request plus the reply you wish the model had given, and the tuning runs on a pile of them.

This is not lesson one’s monumental training-from-scratch but a comparatively small extra round on top of it. It is still real training, though: slower, costlier, and harder to undo than editing a prompt.

What it buys is persistent behavior: your brand voice in every answer, your exact report format, the diction of your specialty. What it is wrong for is facts that change. Nobody re-runs the tuning because the menu got a new latte.

Which lever?

Here is the routing rule that resolves most meetings.

The third row deserves a word, since “not sure” is where most real requests arrive. RAG wins the tie because trying it is cheap. An experiment takes days and touches no weights, so a wrong guess costs a little time, while a wrong fine-tune costs a training run you then have to unwind. When the shape of the problem is unclear, reach for the reversible tool first and let the results tell you what kind of problem you had.

One refinement comes before either lever. The cheapest tool of all is lesson two’s prompt, since instructions cost nothing to change and take effect on the next request. Plenty of requests that arrive labeled “fine-tuning,” brand voice among them, get 80% of the way there with better wording in the system prompt. The free lever comes before the cheap one, and the cheap one before the heavy one.

The mental model to remember

RAG hands the unchanged model the right documents at the right moment: retrieve, augment, generate. It is open books for facts, and hallucination drops because the truth becomes the likeliest thing in sight.

Embeddings (recognition tier) are how the retrieval searches by meaning instead of keywords.

Fine-tuning actually changes the model’s numbers with extra training on your examples, buying persistent behavior at real cost.

The routing rule sends changing facts to RAG and persistent behavior toward fine-tuning, with RAG first when unsure. Before either lever, try better prompt instructions, since the free one comes first.

You should now be able to sit in the meeting where both requests land, one asking “know our weekly menu” and the other “always sound like us,” and route each to its lever without hesitation.