The Adaptation Toolbox
"How do I make a general model good at my task?" has four standard answers, and interviewers love asking you to compare them because the comparison reveals whether you understand what each one actually changes. Two of the four modify the prompt; one modifies the context; one modifies the model itself.
The four tools
1. System prompt — a standing instruction block the developer controls, prepended to every conversation: role, rules, tone, constraints, output format. ("You are a credentialing assistant. Answer only from provided context. Never speculate about eligibility.") The user never sees it and shouldn't be able to override it. It changes behavior, costs nothing, and updates instantly — always the first tool off the shelf.
2. Few-shot examples — worked input→output examples inside the prompt, demonstrating the task. Instead of describing the classification you want, you show three examples of it done right. The model pattern-matches onto the examples — often dramatically better than instructions alone for formatting, classification, and judgment-style tasks. "Zero-shot" means no examples, just instructions; "few-shot" means learning from the demonstration. Key property: examples live in the prompt, so they're versionable text, changeable per request, no training involved.
3. RAG — covered in depth already: inject knowledge into the context at query time. Reach for it when the problem is what the model knows: local, changing, or private information. The decision rule you already have is correct — knowledge that changes belongs in an index you can update, not in model weights.
4. Fine-tuning — the only one that changes the model. You continue training on hundreds or thousands of example pairs, producing new weights. What it's actually good at teaching: style, format, and consistent behavior — your exact document conventions, a specialized output structure, a tone. What it's bad at: adding facts. Facts fine-tuned into weights are frozen (stale the day your policy changes), unverifiable (no citation possible), and unevenly absorbed. It's also the expensive option: training runs, evaluation, hosting a custom model, and re-doing all of it when the base model improves.
The decision ladder
Say it as a ladder and you sound like an architect:
"Prompting first — it's free and instant. Few-shot examples when the task needs demonstration. RAG when the gap is knowledge — anything local, changing, or requiring citations. Fine-tuning last, and only when the gap is behavior that survives good prompting — a format or style the model won't hold — never for knowledge."
The classic fine-tuning-is-wrong case (know this cold): "fine-tune the model on our policy documents so it knows our rules." Wrong three ways — the rules change (weights go stale), you need citations (weights can't cite), and accuracy is unverifiable per-fact. The right design is RAG over the current documents, where updating knowledge means updating an index, and every answer carries its source. Fine-tuning might later earn a place in the same system teaching output format — after RAG solved knowledge.
Interview drill
- Define each of the four in one sentence, including what it changes (behavior / task demonstration / knowledge / weights).
- Recite the ladder.
- Give the policy-documents story as your fine-tuning-is-wrong example, with the three reasons.