RAG End to End, No Gaps

You know retrieval; the commonly missing piece is the G. RAG stands for Retrieval-Augmented Generation: retrieval collects evidence, and then a language model writes the answer from that evidence. Retrieval alone is a search engine. The generation step is what turns search results into a direct, grounded answer — and it's also a second place where things can go wrong. This lesson is the complete pipeline, walkable from memory.

The two phases

Phase 1 — Ingestion (before any user shows up):

1. Collect the authoritative documents (and only authoritative ones — corpus scoping is a governance decision).
2. Chunk them — split into passages, respecting structure.
3. Embed each chunk with the embedding model.
4. Store (vector, chunk text, source metadata — document, section, version, date) in the index. The metadata is what makes citations possible later.

Phase 2 — Query time (every question):

5. Embed the user's question with the same embedding model.
6. Vector search: top-k nearest chunks. Optionally rerank — a second, more careful pass that reorders candidates by true relevance (this is the "retrieve candidates, then verify them" instinct, productized).
7. Assemble the prompt — this is the load-bearing step people forget. The prompt contains: the retrieved chunks (with their source labels), the user's question, and instructions like: "Answer using only the provided context. Cite the source for each claim. If the context doesn't contain the answer, say you don't know."
8. The LLM generates the answer from that prompt — grounded, citing chunks.
9. Return the answer with its citations, so a human can check the claim against the source. In high-stakes systems, log the whole assembly (question, chunks used, answer) — that log is your audit trail.

One-sentence summary for the room: "Retrieval finds the evidence; the LLM writes the answer from the evidence; citations make it checkable."

Every way it produces a wrong answer with a real citation

This question separates people who've operated RAG from people who've read about it. There are failure modes on both sides:

Retrieval side:

• Over-matching — the answer isn't in the corpus, so search returns the nearest related chunk, and the answer gets built on the wrong source. Defense: relevance thresholds + "I don't know" as a first-class outcome.
• Stale corpus — the citation is real, the document is outdated. Defense: corpus versioning and re-indexing discipline.
• Bad chunking — the retrieved fragment lacks the qualifier that changes the meaning ("waived for renewals only"). Defense: chunking that respects document structure.

Generation side (possible even with perfect retrieval):

• The model misreads or blends chunks — merging two requirements into one that doesn't exist, decorated with both citations.
• The model answers from its own weights — ignoring the context and generating from training memory, then attaching a citation cosmetically. Defense: strict "context-only" instructions, low temperature, and evaluation questions designed to catch exactly this.
• The question outruns the context — the user asks for a synthesis the chunks don't support, and the model obliges anyway.

Because failures live on both sides, so does verification: retrieval metrics (are the right chunks found?) and generation metrics (is the answer faithful to the chunks?) are measured separately. That sentence alone signals real practice.

Interview drill

Whiteboard the nine steps from memory — ingestion (collect, chunk, embed, store) and query (embed, search/rerank, assemble prompt, generate, return with citations). Then recite four wrong-with-citation causes: over-match, stale corpus, chunk fragmentation, unfaithful generation — and one defense for each. When you can do both cold, this topic is closed.