Choosing a Strategy

Interviewers rarely ask "what is fine-tuning?" — they describe a scenario and ask which adaptation technique you'd choose and why. This lesson is the decision framework that answers that question every time.

The core rule: knowledge vs behaviour

Classify the problem first:

• Knowledge problem — the model doesn't know something (your policies, your catalogue, this week's prices) → RAG. Retrieval adds facts at query time, stays current, and can cite.
• Behaviour problem — the model knows enough but responds in the wrong shape (format, tone, vocabulary, convention-following) → fine-tune, after prompting and few-shot examples have been exhausted.
• Both → combine them. A model fine-tuned to speak your domain's language, wired into a RAG pipeline for the facts, is a standard enterprise pattern — the techniques are complementary, not rivals.
• Volatile data → never bake it into weights. Anything that changes weekly belongs in a retrieval index that re-ingests on change, not in a model you'd have to retrain on every edit.

The four secondary dimensions

When the core rule leaves two options open, score them on:

• Cost. Prompting costs tokens; RAG costs an ingestion pipeline plus retrieval per request; fine-tuning costs data curation, training runs, and hosting a dedicated deployment. But a tuned model can reduce per-request cost by replacing thousands of prompt tokens with weights.
• Latency. RAG adds an embedding + search hop before generation. A fine-tuned model with a short prompt can be the fastest option — relevant for high-volume, low-latency endpoints.
• Data privacy. Training data leaves your boundary and persists inside a model artefact. With RAG, documents stay in your own store (the Tutor's chunks never leave SQL Server) and you can enforce per-user row-level security at retrieval time — impossible once data is in weights, which every caller of the model can potentially elicit.
• Auditability. RAG shows its evidence; regulated and public-sector systems often require citations. A fine-tuned model cannot show its work.

Worked example: a credential-requirements assistant

Scenario: citizens ask which documents they need for a professional credential. Requirements change with legislation, and wrong answers have real consequences.

Apply the framework: the gap is knowledge, the data is volatile, and answers must be auditable — three independent signals all pointing at RAG. Fine-tuning would freeze last year's requirements into weights with no citations and a retraining bill on every legislative change. If the assistant also had to emit responses in a rigid government letter format, you might add a light fine-tune for that behaviour — but the knowledge still comes from retrieval. State the classification, name the dimensions, give the hybrid caveat: that is the complete interview answer.