Choosing the Offline Metric for Candidate Generation
Problem statement
How would you evaluate the videos a recommender retrieves before ranking them? Several sources search a huge catalog, then a heavier model chooses what users see. You have held-out records of videos users later finished watching. Calculate retrieval coverage and explain how to choose and evaluate a retriever.
Show hint
Think about what the candidate stage is responsible for and what happens to an item it leaves out.
Relevant roles: ML Engineer, Applied Scientist and Data Scientist.