RAG vs Agentic RAG

Overview

Classic RAG is a fixed pipeline. It embeds the question, retrieves top-K chunks once, and generates an answer. Agentic RAG puts an LLM agent in control of retrieval. The agent decides whether it needs to search at all, which source to query, how to rewrite the query, whether the results are good enough, and whether to search again before answering.

The benefit is better answers for complex questions, such as comparisons, multi-step lookups, or questions spanning several systems. The cost is more LLM calls, higher and less predictable latency, and harder debugging. A good design uses classic RAG for simple questions and escalates to the agentic path only when needed.

When to use it

• Questions that need several lookups, for example comparing two policies or joining data across systems.
• Many heterogeneous sources such as docs, SQL databases, APIs, and web search.
• When single-shot retrieval often returns irrelevant chunks.

Where it shows up in interviews

Where it is used in real software

Key terms

Query rewriting

Rephrasing or splitting the question to improve retrieval.

Router

A step that chooses which source or tool to query.

Self-evaluation

The agent judges whether retrieved context is sufficient before answering.

Iteration limit

A cap on retrieval rounds to bound cost and latency.

How it works, step by step

1. 1
   Decide whether to retrieve

   Simple greetings or general questions may not need retrieval at all.

2. 2
   Plan and route

   Break the question into sub-questions and choose a tool for each, such as vector search, SQL, or an API.

3. 3
   Retrieve and grade

   Run searches and score whether results are relevant and sufficient.

4. 4
   Refine or continue

   If results are weak, rewrite the query or try another source, up to an iteration limit.

5. 5
   Synthesize with citations

   Combine the gathered evidence into one answer that cites each source.

Classic vs agentic

Same knowledge base, three question types.

Route simple questions to classic RAG and escalate complex ones to the agent.

Implementation

1type Tool = { name: string; run: (query: string) => Promise<string[]> };2 3async function agenticRag(question: string, tools: Tool[], maxRounds = 3) {4  const evidence: string[] = [];5  let query = question;6 7  for (let round = 0; round < maxRounds; round++) {8    const decision = await llmJson<{ tool: string; query: string } | { done: true }>(9      `Question: ${question}\nEvidence so far:\n${evidence.join("\n")}\n` +10      `Tools: ${tools.map((t) => t.name).join(", ")}. Reply {"done":true} if evidence is sufficient, ` +11      `otherwise {"tool": name, "query": text}.`,12    );13    if ("done" in decision) break;14    const tool = tools.find((t) => t.name === decision.tool);15    if (!tool) break;16    query = decision.query;17    evidence.push(...(await tool.run(query)));18  }19 20  return llmText(`Answer using only this evidence and cite it:\n${evidence.join("\n")}\n\nQuestion: ${question}`);21}

Complexity and performance

Trade-offs

Variants and related techniques

Common mistakes

• Unbounded loops.

  Fix: Set a maximum number of rounds, a token budget, and a timeout.

• Using the agent for every question.

  Fix: Route by complexity so most traffic stays on the cheap, fast path.

• No tracing.

  Fix: Log every plan, tool call, and retrieved result to debug and evaluate behavior.

Interview questions

Practice problems