Building, Calibrating and Debugging a CTR Classifier
Problem statement
How would you predict clicks on promoted products in a shopping app? Clicks are rare, traffic is very high, and predictions are multiplied by advertiser bids to order the slots. The probabilities need to be trustworthy, and you must be able to investigate a performance drop after launch.
Walk through data, features, models, training, calibration, evaluation, deployment and debugging.
Show hint
Ask what the model's output is used for, and therefore which property of the prediction matters most besides ordering.
Relevant roles: ML Engineer, Applied Scientist and Data Scientist.
Related free problems
- Debugging a Transformer Text Classifier's Failing TestsDeep LearningHard
- Debugging a Patient-Message Triage ClassifierML System DesignMedium