Language models
Retrieval and RAG
Like answering an open-book question, a RAG system looks up relevant material before writing. Retrieval-augmented generation places selected source passages in a language model’s context. The answer can use current or private information without storing every fact in model weights.
How it works
- Split source documents into chunks and store text, identifiers, access rules, and retrieval features.
- Search the permitted chunks using lexical matching, embeddings, or both.
- Optionally rerank the candidates and fit useful passages into the context budget.
- Ask the generator to answer from the evidence, cite sources, and abstain when evidence is insufficient.
- Evaluate retrieval recall separately from answer correctness, grounding, and citation support.
The knobs that matter
- chunk size and overlap
- trade local context against precise retrieval and duplication
- retrieval count
- more candidates can improve coverage but add cost and possible noise
- retrieval method
- lexical and dense search capture different kinds of matches
- reranker
- adds scoring work to improve which passages reach generation
What interviewers ask
- How would you debug a retrieval chatbot that gives confident but wrong answers?
- How do you choose chunk size and evaluate retrieval separately from generation?
- When would you use retrieval instead of fine-tuning for a document knowledge base?
- How would you implement citations and a no-answer fallback?
Common mistakes
- Measuring only fluent answers while ignoring whether the right evidence was retrieved.
- Assuming a citation guarantees that the cited passage supports the claim.
- Retrieving unauthorized documents and relying on the generator to hide them.
When to use it
Use RAG when answers depend on searchable documents that change or need citations. It helps less when the task primarily needs a new behavior, or when the required evidence cannot be retrieved reliably.
Implement it (2 problems)
- BM25 RankingProMediumSolve
Learn it
Interview questions about retrieval and rag
- Chunking Strategies for RAGFreeMediumSolve