Building a Grounded Q&A Tool on a Hosted LLM API

MediumLLMs & GenAI

Reported in interviews at Mistral AI

Problem statement

How would you build a document-grounded question-answering tool in a short live coding round? You have a hosted language model, an embeddings endpoint and a folder of internal markdown documents on your laptop. AI coding help is allowed. The tool must cite source files or say it does not know.

Explain API handling, untrusted documents and the complete retrieval pipeline.

Show hint

Which failures of a remote API would succeed if you simply tried again a moment later, and which would fail the same way every time?

Relevant roles: ML Engineer, AI Engineer, LLM Engineer, Agentic AI Engineer and Gen AI Engineer.

Related free problems