Chapter 6: LLM applications
Hybrid retrieval and reranking: BM25, RRF and cross-encoders
Real users type vague descriptions and exact codes, and one search method alone misses one of the two. Interviewers often ask how to fix a search that misses exact product codes. They expect one recipe: keyword search next to dense search, the two lists fused by rank, then a careful reranker on the top results.
What you will learn
- Implement BM25 scoring
- Fuse ranked lists with reciprocal rank fusion
- Explain bi-encoder vs cross-encoder accuracy and cost
Key terms
- BM25
- a keyword score that counts word matches, with two repairs
- RRF
- reciprocal rank fusion, which merges two lists by rank, not by score
- cross-encoder
- a model that reads a query and a document together
- reranking
- sorting a short list again with a slower, sharper model
Prerequisites
Take Embeddings and dense retrieval first. It comes right before this lesson in the course.
Where this lesson picks up: You turned queries and documents into vectors with a two-tower model and found the closest documents by cosine. But a search by meaning can rank an exact product code low. How do you keep the strengths of keyword search too?
What comes next
The fused search is accurate, but a comparison of the query with millions of vectors is slow, so how do you search them fast?
Next lesson: ANN indexes: HNSW and IVF-PQ trade-offs
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.