Chapter 6: LLM applications
RAG vs fine-tuning vs long context
Interviews for AI system design often ask: "RAG, fine-tuning or a long context window?" A strong answer compares freshness, cost, speed and citations with numbers, and says how you test that a long window really works.
What you will learn
- Compare the three options on freshness, cost, latency and citations
- Estimate token cost of stuffing a corpus into context
- Explain lost-in-the-middle and how to test it
Key terms
- RAG
- the model reads chunks it retrieves from an index before it answers
- fine-tuning
- more training on labeled input-output pairs, which changes the weights
- long context
- a very long prompt that holds a whole document
- token
- a word or a part of a word; the unit a model reads and counts
- KV cache
- the stored keys and values for every token the model has read
Prerequisites
Take RAG pipeline: chunking and indexing first. It comes right before this lesson in the course.
Where this lesson picks up: You split documents into chunks and stored them, and you saw that how you chunk and index decides what RAG can ever retrieve. But why not train the facts into the model, or paste every document into the prompt?
What comes next
You picked RAG, so how do you prove that it answers well, and that its answers stay good over time?
Next lesson: Golden sets and LLM-as-judge
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.