Why LLMs Hallucinate: Diagnosing Four Failure Patterns

MediumLLMs & GenAI

Reported in interviews at Zillow

Problem statement

Why does a housing assistant give confident answers that are wrong or unsupported? It handles mortgage terms, local rules and listing details using a language model whose training data is stale. The product team wants to understand how training, data and generation cause these errors so it can choose practical fixes.

Explain the observed error patterns and how practical mitigations address their causes.

Show hint

For each observation, what would the evaluation look like if the cause were decoding, context, training data or training procedure?

Relevant roles: ML Engineer, LLM Engineer, Gen AI Engineer and AI Engineer.

Related free problems