Debugging a RAG Chatbot That Is Confidently Wrong
Problem statement
An internal help chatbot answers employees' questions about workplace policies with retrieval-augmented generation: it embeds each question, retrieves the top 5 chunks from an index of policy pages, and asks an instruction-tuned LLM to answer from them. Users report answers that sound certain but are wrong. The team has collected 200 such failures and labeled each with the correct answer and, where one exists, the gold supporting chunk.
Show hint
Ask why a model trained on next-token likelihood would sound equally sure when right and wrong.
Relevant roles: AI Engineer, Gen AI Engineer and AI Architect.