Chapter 6: LLM applications
Prompt injection, jailbreaks and guardrail layers
You will be able to tell where an attack enters an app, and plan layered defenses against it. Any text a model reads can act as an instruction, so an app with email or web tools can turn against its users. Interviewers ask about this often, and they want layers, not "add a filter".
What you will learn
- Distinguish direct from indirect prompt injection
- Design input, output and action-level guardrails
- Explain why no single filter stops injection
Key terms
- system prompt
- The app's own hidden instruction text.
- prompt injection
- Text that the model reads and follows as a new instruction.
- direct injection
- The user types the attack into the chat.
- indirect injection
- The attack hides in outside content the model reads.
- outside content
- A web page, file, fetched document or image the model reads.
Prerequisites
Take Structured output: JSON schemas, constrained decoding and repair first. It comes right before this lesson in the course.
Where this lesson picks up: You made a model return JSON, and a validator checked that the JSON had the right shape. But a validator checks the shape of an answer, not whose instructions the model followed.
What comes next
Next, the app fetches documents by meaning and pastes them into the prompt, and every one of them is text the model will read.
Next lesson: Embeddings and dense retrieval
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.