Chapter 7: ML system design and experiments
The ML system design skeleton
Most ML engineer interview processes have one round where you design a whole ML product in about 45 minutes. Interviewers grade the structure more than the model you name. You will run one six-stage plan on any prompt, pick metrics that match the goal, and launch the model in safe steps.
What you will learn
- Run the skeleton on an unfamiliar prompt in order
- Pick offline and online metrics that match the business goal
- Name rollout (shadow, canary, A/B) and monitoring steps
Key terms
- design round
- An interview round where you plan a whole ML product out loud.
- stage
- One part of that plan, done in a fixed order.
- frame
- To turn the goal into an input, an output and a task type.
- label
- The outcome the model learns to predict, such as whether a user installed an app.
- feature
- An input fact the model uses, such as past installs or app category.
Prerequisites
Take Workflows vs agents: planning, memory and multi-agent orchestration first. It comes right before this lesson in the course.
What comes next
Your plan says "rank apps for every user, fast": how many requests per second is that, and how many milliseconds does each stage get?
Next lesson: Back-of-envelope: QPS, storage and latency budgets
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.