Agentic RAG — Retrieval as a Tool, Not a Reflex

Module 6 taught the RAG pipeline: retrieve, then generate. That pattern is right for a focused knowledge assistant, but it becomes a liability the moment your system is an agent with its own tools. The upgrade is agentic RAG: the model decides whether, what, and how to retrieve — instead of running a vector lookup on reflex before every answer.

Contrast diagram showing reflexive vector retrieval forced before every answer versus an agent choosing among semantic search, grep, and file reading as tools.

The anti-pattern: retrieval on every turn

A common mistake is to make semantic retrieval a fixed pipeline stage — every question triggers a top-k vector search whose chunks are injected into the prompt. This doesn't just waste a step; it anchors the model. If the retrieved chunks are irrelevant (very common for precise or code questions), the model is now biased toward wrong context and answers worse than if it had simply looked. An agent that already has search, file-read, and reference-following tools does not need a reflexive vector lookup — it needs the judgment to pick the right tool.

Retrieval is one tool among several

Reframe retrieval as a tool the agent calls (Module 7), alongside keyword/grep search and direct reads. Then give the agent a routing policy:

Semantic index — when the query is fuzzy or conceptual and you don't know the exact terms to search for. Recall when you can't grep.
Lexical search + read + follow-references — when you know exact symbols, routes, or files, or the question is a "how is X handled" trace across components. This covers most code and precise questions.
Hybrid — use semantic search to locate candidate regions, then grep and read to verify and trace the details.

The agent should be free to issue multiple searches, mix lexical and semantic, and prefer reading real source over trusting a retrieved snippet.

Why this matters

Naive "retrieve-then-generate" assumes the answer lives in one chunk. Real questions are often multi-hop — the answer spans a controller, a service, a view, and a config file — which a single vector lookup can never assemble. An agent that reasons ("what do I need? where would it live?"), fetches it the best way, and iterates will outperform a precomputed index on exactly these questions. It's why capable coding assistants lean on agentic navigation and treat embeddings as one signal, not the engine.

Steering it in practice

A prompt steers tool choice but doesn't guarantee it — models don't always route well. So bias hard in the system instructions ("do not default to the index; for specific symbols or 'how is X implemented,' search and read the files"), and if it still over-retrieves, add a light heuristic gate and measure which path actually answers correctly. Agentic RAG is the bridge between Module 6 (RAG), Module 7 (Tool Calling), and this module: retrieval is a tool, not a pipeline stage.