Tool-Calling Weather Agent Loop

MediumLLMs & GenAI

Reported in interviews at The Voleon Group

Problem statement

A user asks your assistant: "What is the temperature in New York today?" A language model cannot know today's weather from its weights, so the assistant must let the model request a weather lookup, run that lookup itself against an authorized data source, hand the result back, and only accept a final answer that is grounded in what the tool returned.

In the live version of this exercise you would call a hosted chat-completions API with a tool schema. To make the orchestration testable, the model is replaced by a scripted list of replies: each time your loop would call the model, it takes the next reply from model_replies. Your job is everything around the model: validating its tool calls, executing them, keeping provider data from leaking into the conversation, enforcing a budget, and checking the final answer.

def run_weather_agent(model_replies: list, weather_db: dict, max_tool_calls: int) -> dict:
    ...

Model replies (consumed in order). Each reply is one of:

  • {"type": "tool_call", "name": str, "arguments": str}, where arguments is a JSON-encoded string, exactly as chat APIs return it.
  • {"type": "final", "content": str}.

The tool. The only tool is get_current_weather(city, unit). Execute a tool call as follows and append its result to tool_results:

  1. If name is not "get_current_weather", the result is {"error": "unknown_tool"}.
  2. Parse arguments. The result is {"error": "invalid_arguments"} if it is not valid JSON, is not a JSON object, contains any key other than city and unit, has a city that is missing, not a string, or blank after trimming, or has a unit other than "celsius" or "fahrenheit" (unit is optional and defaults to "celsius").
  3. Normalize the city by lower-casing, trimming, and collapsing runs of whitespace to one space, then look it up in weather_db. A missing key gives {"error": "unknown_city"}; a record that contains the key "error" gives {"error": "provider_error"}.
  4. Otherwise the record has name, temp_c, observed_at and possibly other fields. Return exactly {"city": name, "temperature": t, "unit": unit, "observed_at": observed_at}, where t is the temperature in the requested unit ($F = C \cdot 9/5 + 32$) rounded with round(t, 1). No other provider field may be copied into the result.

Loop rules.

  • If the model requests a tool call when max_tool_calls calls have already been executed, stop with status "tool_limit".
  • A final reply ends the loop. It is accepted, with status "answered" and answer equal to the content, only if at least one call has succeeded and the content contains the most recent successful temperature formatted with one decimal ("%.1f" % t). Otherwise the status is "ungrounded".
  • If the replies run out before a final reply, the status is "no_answer".

Return {"status": str, "answer": str or None, "tool_results": list}, with answer set to None unless the status is "answered".

Constraints

  • 0 <= len(model_replies) <= 10, 0 <= max_tool_calls <= 6, weather_db has at most 10 records.
  • Temperatures in weather_db have one decimal place, between $-50$ and $50$ degrees Celsius.

Example 1

weather_db = {"new york": {"name": "New York, NY, US", "temp_c": 21.4, "observed_at": "2026-06-02T14:00:00Z"}}, max_tool_calls = 3, and the replies are a call with arguments '{"city": "New York", "unit": "fahrenheit"}' followed by the final text "It is 70.5 F in New York right now (observed 14:00 UTC)."

Output: status "answered", that text as the answer, and tool_results = [{"city": "New York, NY, US", "temperature": 70.5, "unit": "fahrenheit", "observed_at": "2026-06-02T14:00:00Z"}], because $21.4 \cdot 9/5 + 32 = 70.52$.

Example 2

Same database, but the only reply is the final text "It is usually about 75 F in New York in June." Output: status "ungrounded", answer None, tool_results = []. The model answered from memory without calling the tool.

Example 3

Same database. The replies are a call with the truncated arguments '{"city": "New York", "unit": "fahrenheit"', then a call with '{"city": "New York"}', then the final text "Currently 21.4 C in New York." Output: status "answered", with tool_results = [{"error": "invalid_arguments"}, {"city": "New York, NY, US", "temperature": 21.4, "unit": "celsius", "observed_at": "2026-06-02T14:00:00Z"}].

Show hint

What part of this system should the model decide, and what part should deterministic code decide and verify?

Relevant roles: ML Engineer, AI Engineer, Agentic AI Engineer, LLM Engineer and Gen AI Engineer.

Related free problems