Chapter 7: ML system design and experiments

Ranking and re-ranking: multi-objective, diversity, cold start

A feed must put good videos first, keep the list varied and fresh, and still give new videos a chance. Every recommendation design interview reaches this point: "You have a few thousand candidates. Now order them."

21 minInteractive lessonChapter 7: ML system design and experimentsLesson 8 of 13

What you will learn

  • Combine multiple predicted objectives into a ranking score
  • Apply diversity and freshness re-ranking
  • Handle cold start with exploration such as Thompson sampling

Key terms

objective
one outcome the model predicts for a video, such as a click
re-ranking
a last step that changes the order of the ranked list
cold start
a new video with no click history yet
regret
clicks lost compared with always showing the best video

Prerequisites

Take Candidate generation: two-tower and matrix factorization first. It comes right before this lesson in the course.

Where this lesson picks up: You pulled a few hundred to a few thousand candidate videos out of millions with a two-tower model. That pile has no order yet, so which video goes first?

What comes next

Every candidate you score costs computer time, so the next lesson asks what one request costs.

Next lesson: BOTEC for LLM products: tokens, GPUs and cost per request

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.