Evaluating RAG Systems, RAG Components and the Transformer
Problem statement
How would you evaluate an assistant that answers employee questions from internal documents? It retrieves passages, gives them to a model and cites sources in its answers. You need to tell retrieval failures from answer-writing failures and evaluate a new customer deployment with no labeled data.
Explain stage-level evaluation and how to test quality when labels are missing.
Show hint
Picture the pipeline as stages and ask where each kind of failure would originate.
Relevant roles: AI Engineer, Gen AI Engineer, LLM Engineer, AI Architect and ML Engineer.