Chapter 6: LLM applications

ANN indexes: HNSW and IVF-PQ trade-offs

Exact search compares a query with every stored vector, so at millions of vectors it is too slow and too big. Every retrieval design interview asks how you find the nearest vectors in milliseconds. Interviewers expect two index families by name, HNSW and IVF-PQ, and today you build both from small pieces.

7 minInteractive lessonChapter 6: LLM applicationsLesson 5 of 12

What you will learn

  • Explain how HNSW searches a layered graph
  • Compute IVF-PQ memory per vector
  • Tune recall against latency with efSearch or nprobe

Key terms

recall@10
the share of the true 10 nearest vectors that an index returns
HNSW
a layered graph of links that you walk from the top layer down
IVF-PQ
clusters of vectors, with each vector stored as a few one-byte codes
nprobe
how many clusters an IVF search opens

Prerequisites

Take Hybrid retrieval and reranking: BM25, RRF and cross-encoders first. It comes right before this lesson in the course.

Where this lesson picks up: You fused the BM25 list and the dense list, and a slow cross-encoder reranked only the top candidates. But the dense retriever must first find the 100 nearest vectors among millions, and fast: how does it do that?

What comes next

We can find neighbors fast; next, what exactly do we store as one vector?

Next lesson: RAG pipeline: chunking and indexing

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.