Designing an LLM Solver for Arithmetic Questions

MediumLLMs & GenAI

Reported in interviews at Tesla

Problem statement

How would you build an LLM service that answers arithmetic questions reliably? Some inputs are plain expressions, some fit known formulas, and others need several steps. The service can use a safe parser, a formula template or sandboxed code, with different speed and reliability trade-offs.

Explain routing, tool validation, verification and safeguards, along with prompts, evaluation and experiment tracking.

Show hint

Which part of answering "sum from 1 to 100" actually needs language understanding, and which part only needs exact computation?

Relevant roles: ML Engineer, AI Engineer, LLM Engineer and Agentic AI Engineer.

Related free problems