Multi-Tenant Enterprise RAG Under Freshness and Cost Limits

HardLLMs & GenAI

Reported in interviews at Microsoft OpenAI

Problem statement

How would you build a document question-answering service for many enterprise customers? It must handle a large, changing, multilingual corpus and make edits searchable quickly. Answers must be fast and affordable, protect private information, and respect both customer boundaries and document permissions.

Size the index, enforce retrieval permissions and diagnose stale answers.

Show hint

Which requirements are hard correctness constraints rather than quality goals, and how does each one shape where data lives and how it is searched?

Relevant roles: ML Engineer, AI Engineer, LLM Engineer, Gen AI Engineer and AI Architect.

Related free problems