Chapter 6: LLM applications
Embeddings and dense retrieval
You will score videos against a query by cosine and train the two networks that make those scores. Many search engines, recommenders and look-up assistants start with this step, called retrieval. Interviews ask you to design it, write its loss and say what goes out of date.
What you will learn
- Embed, normalize and search a small corpus by cosine similarity
- Explain how two-tower models are trained with in-batch negatives
- Explain domain shift and stale embeddings
Key terms
- Cosine
- how closely two arrows point one way: 1 same, 0 right angles, −1 opposite
- Length
- the square root of the sum of the squared numbers
- Normalize
- divide each number by the length, so the new length is 1
- Dot product
- multiply matching numbers, then add the results
Prerequisites
Take Prompt injection, jailbreaks and guardrail layers first. It comes right before this lesson in the course.
Where this lesson picks up: You learned that any text the model reads can carry instructions, so retrieved documents must be treated as data. That leaves a question: out of millions of documents, how do you pick the few to fetch?
What comes next
Search by meaning finds "my ideal car" but can miss an exact product code, so what catches it?
Next lesson: Hybrid retrieval and reranking: BM25, RRF and cross-encoders
More lessons in this part
- Hybrid retrieval and reranking: BM25, RRF and cross-encoders 22 min
- ANN indexes: HNSW and IVF-PQ trade-offs 7 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.