Chapter 5: Reinforcement learning and post-training
Prompt, RAG or fine-tune?
You will be able to pick prompting, retrieval or fine-tuning from what a model lacks. Interviews often open with this choice. A strong answer names the missing piece and what each fix costs.
What you will learn
- Pick prompting, RAG, fine-tuning or a mix from a scenario
- Name the data and cost needed for each option
- Explain catastrophic forgetting and distillation as alternatives
Key terms
- foundation model
- a language model pre-trained on huge amounts of text
- prompt engineering
- changing the instructions and examples you type into a model
- parameter
- a number inside the model that training sets
- shot
- one example placed in a prompt
Prerequisites
Take Thompson sampling: picking by sampling your beliefs first. It comes right before this lesson in the course.
Where this lesson picks up: You picked slot machines by what you still did not know. Today you pick how to adapt a language model by what it lacks.
What comes next
You chose to fine-tune: what does the model train on, and which tokens count in the loss?
Next lesson: SFT: chat templates and masked loss
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.