RAG & VECTOR DATABASES / SYSTEM CONCEPT BRIEF

RAG vs fine-tuning

RAG and fine-tuning change different things.

IntermediatePhase 10 / Topic 23 of 25RequirementsTrade-offsFailure modes
01

Overview

RAG and fine-tuning change different things. RAG changes what the model knows at request time by putting relevant documents into the prompt. Fine-tuning changes how the model behaves by further training its weights on examples. A useful rule is that RAG supplies knowledge while fine-tuning teaches skills, style, and format.

Most teams should start with prompting, add RAG when answers need private or changing facts, and consider fine-tuning only when the model still fails at a repeatable behavior, such as a strict output format, a domain writing style, or a narrow classification task. The two can also be combined, with a fine-tuned model reading retrieved context.

Reference binder vs training course

RAG is handing a new employee a binder of current policies to look things up. Fine-tuning is sending them on a training course so they naturally write and act the company way. The binder is easy to update tomorrow; the course changes habits but gets out of date when policies change.

02

When to use it

  • Deciding how to adapt an LLM to a company's data or domain.
  • Explaining cost, freshness, and accuracy trade-offs to stakeholders.
  • Planning an AI roadmap from prototype to production.
03

Where it shows up in interviews

Knowledge vs behavior

Recognize it when: the interviewer asks how to make the model know company information.

  • Design a RAG system
  • Design an AI customer-support system
Cost and latency optimization

Recognize it when: a large model works but is too slow or expensive.

  • Design an LLM inference platform
  • Design an AI coding assistant
04

Where it is used in real software

Policy and documentation assistants

Use RAG because content changes weekly and answers need citations.

Structured extraction

Teams fine-tune small models to output a precise JSON schema from invoices or forms at low cost.

Brand voice

Marketing tools fine-tune on approved copy so drafts match tone without long style instructions.

05

Key terms

Fine-tuning
Continuing training of a pre-trained model on task-specific examples.
LoRA
A parameter-efficient method that trains small adapter matrices instead of all weights.
Knowledge cutoff
The date after which the model has no training data.
Catastrophic forgetting
Losing general ability when fine-tuning pushes the model too far toward narrow data.
06

How it works, step by step

  1. 1
    Try prompting first

    Clear instructions and a few examples solve many problems with no infrastructure.

  2. 2
    Add RAG for facts

    If answers depend on private, large, or changing information, retrieve it into the prompt.

  3. 3
    Measure the remaining failures

    Build an evaluation set and classify errors as missing knowledge or wrong behavior.

  4. 4
    Fine-tune for behavior

    If failures are about format, tone, or a narrow skill, fine-tune with a few hundred to thousands of high-quality examples.

  5. 5
    Combine when needed

    A fine-tuned model can still use RAG, getting consistent behavior and fresh facts.

Choosing an adaptation path
Step 1 / 5
Prompting
RAG
Evaluate
Fine-tune
Combine

STEP 1Start with instructions and examples. It is the cheapest and fastest to change.

07

Side-by-side comparison

Adapting a model for an insurance company.

Step 1 / 6
FactorRAGFine-tuning
Adds new factsYes and instantlyPoorly and goes stale
Changes style or formatSomewhat through promptYes and reliably
CitationsNaturalNot available
Upfront costIndexing pipelineTraining data and compute
Per-request costLonger promptsShorter prompts and possibly smaller model
Access controlFilter documents per userHard since knowledge is in weights

NOWFactor: Adds new facts | RAG: Yes and instantly | Fine-tuning: Poorly and goes stale

Use RAG for policy answers and fine-tune only for the claim-summary format the adjusters need.

08

Implementation

type Failure = { kind: "missing-fact" | "stale-fact" | "wrong-format" | "wrong-tone" | "reasoning" }; function recommend(failures: Failure[]): string[] {  const count = (k: Failure["kind"]) => failures.filter((f) => f.kind === k).length;  const plan: string[] = [];  if (count("missing-fact") + count("stale-fact") > 0) plan.push("Add or improve RAG retrieval");  if (count("wrong-format") + count("wrong-tone") > failures.length * 0.2) plan.push("Fine-tune (LoRA) on curated examples");  if (count("reasoning") > 0) plan.push("Try a stronger model or break the task into steps");  return plan.length ? plan : ["Keep prompting; failures are rare"];}
09

Complexity and performance

RAG update timeminutes

Re-embed changed documents.

Fine-tune update timehours

Retrain and redeploy the model.

Data needed for fine-tuninghundreds to thousands of examples

Quality matters more than quantity.

10

Trade-offs

Freshness vs consistency

RAG is always current but depends on retrieval quality. Fine-tuning gives consistent behavior but bakes in a snapshot.

Prompt length vs training cost

RAG adds tokens to every request. Fine-tuning costs upfront but can shorten prompts and allow a smaller model.

11

Variants and related techniques

Parameter-efficient fine-tuning

LoRA and similar methods train a small fraction of weights, making tuning cheaper and adapters swappable.

Retrieval-aware fine-tuning

Train the model on examples that include retrieved context so it learns to use and cite it well.

12

Common mistakes

  • Fine-tuning to teach facts.

    Fix: Facts in weights go stale and cannot be cited; keep facts in a retrievable store.

  • Fine-tuning before building an evaluation set.

    Fix: Measure the baseline first so you can prove the tuned model is better.

13

Interview questions

A company wants the model to know its product catalog. RAG or fine-tuning?

RAG. The catalog changes, answers need exact prices and availability, and retrieval allows citations and per-region filtering. Fine-tuning would bake in stale data and still hallucinate details.

When is fine-tuning the right choice?

When the model must reliably follow a specific format, style, or narrow skill that prompting cannot achieve consistently, or when you want a smaller, cheaper model to match a larger one on a focused task.

14

Practice problems

ProblemDifficultyWhat it trains
Classify ten product requirements as RAG, fine-tuning, or bothEasyKnowledge vs behavior.
Design an adaptation plan for a legal document assistantMediumEvaluation-driven decisions.