Chapter 6: LLM applications

RAG vs fine-tuning vs long context

Interviews for AI system design often ask: "RAG, fine-tuning or a long context window?" A strong answer compares freshness, cost, speed and citations with numbers, and says how you test that a long window really works.

20 minInteractive lessonChapter 6: LLM applicationsLesson 7 of 12

What you will learn

  • Compare the three options on freshness, cost, latency and citations
  • Estimate token cost of stuffing a corpus into context
  • Explain lost-in-the-middle and how to test it

Key terms

RAG
the model reads chunks it retrieves from an index before it answers
fine-tuning
more training on labeled input-output pairs, which changes the weights
long context
a very long prompt that holds a whole document
token
a word or a part of a word; the unit a model reads and counts
KV cache
the stored keys and values for every token the model has read

Prerequisites

Take RAG pipeline: chunking and indexing first. It comes right before this lesson in the course.

Where this lesson picks up: You split documents into chunks and stored them, and you saw that how you chunk and index decides what RAG can ever retrieve. But why not train the facts into the model, or paste every document into the prompt?

What comes next

You picked RAG, so how do you prove that it answers well, and that its answers stay good over time?

Next lesson: Golden sets and LLM-as-judge

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.