Chapter 6: LLM applications
Structured output: JSON schemas, constrained decoding and repair
Code that reads a model's reply breaks when the reply is not valid JSON. You will be able to make an invalid reply impossible, not only rare, and to compute one such step by hand. Interviewers ask this, and they expect more than a better prompt.
What you will learn
- Specify an output contract as a JSON schema
- Explain how grammar-constrained decoding guarantees validity
- Build a validate-and-repair loop with a retry budget
Key terms
- JSON
- a text format that stores data as named fields, so code can read it
- JSON schema
- the contract for a reply: each field, its type, and which fields are required
- logit
- the model's raw score for one possible next token
Prerequisites
Take RL pitfalls: all-fail groups, entropy collapse and contaminated evals first. It comes right before this lesson in the course.
What comes next
The reply is now valid JSON every time, but what if the text the model read told it to put something harmful inside that JSON?
Next lesson: Prompt injection, jailbreaks and guardrail layers
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.