Designing a General-Purpose LLM Chat Assistant

HardLLMs & GenAI

Reported in interviews at Meta Amazon

Problem statement

How would you build a general-purpose conversational assistant? It must sustain dialogue, use tools, ground answers in fresh or private information, and refuse harmful requests appropriately. Enterprise data must stay private and out of training. Responses stream quickly, with a small fast model handling most traffic and a larger model handling harder requests.

Explain preference-training failures, private grounding and production serving.

Show hint

Separate what the assistant knows, how it behaves, and what it can look up: which stage of the system owns each?

Relevant roles: ML Engineer, LLM Engineer, Gen AI Engineer and AI Engineer.

Related free problems