Chapter 6: LLM applications
ANN indexes: HNSW and IVF-PQ trade-offs
Exact search compares a query with every stored vector, so at millions of vectors it is too slow and too big. Every retrieval design interview asks how you find the nearest vectors in milliseconds. Interviewers expect two index families by name, HNSW and IVF-PQ, and today you build both from small pieces.
What you will learn
- Explain how HNSW searches a layered graph
- Compute IVF-PQ memory per vector
- Tune recall against latency with efSearch or nprobe
Key terms
- recall@10
- the share of the true 10 nearest vectors that an index returns
- HNSW
- a layered graph of links that you walk from the top layer down
- IVF-PQ
- clusters of vectors, with each vector stored as a few one-byte codes
- nprobe
- how many clusters an IVF search opens
Prerequisites
Take Hybrid retrieval and reranking: BM25, RRF and cross-encoders first. It comes right before this lesson in the course.
Where this lesson picks up: You fused the BM25 list and the dense list, and a slow cross-encoder reranked only the top candidates. But the dense retriever must first find the 100 nearest vectors among millions, and fast: how does it do that?
What comes next
We can find neighbors fast; next, what exactly do we store as one vector?
Next lesson: RAG pipeline: chunking and indexing
More lessons in this part
- Embeddings and dense retrieval 20 min
- Hybrid retrieval and reranking: BM25, RRF and cross-encoders 22 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.