Vocabulary: Retrieval and RAG

Embedding — a fixed-length vector of numbers representing a text's meaning, produced by a dedicated embedding model, such that similar meanings land close together. Lives in: the retrieval layer (outside the LLM). Say it: "Keyword search matches strings; embedding search matches meaning — 'reset my password' finds 'can't log in.'"

Embedding model — the separate, smaller model that converts text to vectors. The same model must embed both the documents and the queries, or nothing matches. Lives in: the retrieval layer. Say it: "Changing the embedding model means re-embedding the whole corpus — that's a planned migration, not a config flip."

Vector database / index — a store of (vector, text, metadata) entries that answers one question fast: which stored items are nearest to this query vector (top-k nearest neighbors). Lives in: the retrieval layer; can be a dedicated product or a capability inside SQL Server or Postgres. Say it: "SQL Server 2025 has native vector support, so retrieval can live inside the database we already govern."

Chunking — splitting documents into passages before embedding. Too small loses meaning; too large blurs topics. Respecting document structure measurably improves retrieval. Lives in: ingestion (the pipeline's preparation phase). Say it: "Half of retrieval quality is decided at chunking time, before any query exists."

Ingestion — the preparation pipeline: collect authoritative documents, chunk, embed, store with source metadata. Runs before users ever ask anything. Lives in: the RAG pipeline, phase one. Say it: "Ingestion is where corpus governance happens — only authoritative, versioned documents get in."

Corpus — the body of documents a retrieval system draws from. Its scope and freshness bound everything the system can correctly say. Lives in: data governance meets retrieval. Say it: "The system is only as current as its corpus — so re-indexing on document change is part of the design."

RAG (retrieval-augmented generation) — the pattern: embed the question, retrieve the most relevant chunks, assemble them into the prompt, and have the LLM generate an answer from that evidence, with citations. Lives in: the standard architecture for grounded Q&A. Say it: "Retrieval finds the evidence; the model writes the answer from the evidence; citations make it checkable."

Grounding — constraining the model to answer from supplied context rather than its training memory. Lives in: generation-side discipline. Say it: "Grounding is what turns 'the AI said so' into 'the AI cited section four, and here it is.'"

Citation — the source reference attached to each claim, pointing at the exact document and section (and version) the answer relied on. Lives in: auditability. Say it: "For credentialing answers, citations aren't a nicety — they're what makes AI output reviewable."

Top-k — the number of nearest chunks retrieval returns as candidates for the prompt. Lives in: retrieval tuning. Say it: "We tune top-k against the context budget — more chunks isn't better if relevance dilutes."

Relevance threshold — the minimum similarity score below which the system refuses to answer instead of running with the least-bad match. Lives in: retrieval-side safety. Say it: "Below the threshold, the right answer is 'I don't know, here's a human' — never the nearest miss."

Over-matching — the failure where the real answer isn't in the corpus, so retrieval returns the closest related chunk and the answer gets built on the wrong source. Lives in: retrieval failure modes. Say it: "Vector search always returns something — over-matching is why we set thresholds and test with questions the corpus can't answer."

Reranking — a second, more careful pass that reorders the retrieved candidates by true relevance before prompt assembly. Lives in: retrieval quality; the productized version of "retrieve candidates, then verify." Say it: "Cheap search gets candidates; the reranker decides what actually enters the context."

Semantic search vs. keyword search — meaning-based (embeddings) versus string-based matching; production systems often run both (hybrid search). Lives in: search architecture. Say it: "We run hybrid — keyword for exact identifiers, semantic for concepts — because each misses what the other catches."

Corpus versioning / freshness — knowing which edition of every document is indexed, re-indexing on change, and exposing as-of dates in answers. Lives in: data governance; the fix for stale-citation failures. Say it: "Every answer carries the source's as-of date, so even a staleness window is visible instead of silent."