Building, Calibrating and Debugging a CTR Classifier

HardClassical ML

Reported in interviews at Apple

Problem statement

How would you predict clicks on promoted products in a shopping app? Clicks are rare, traffic is very high, and predictions are multiplied by advertiser bids to order the slots. The probabilities need to be trustworthy, and you must be able to investigate a performance drop after launch.

Walk through data, features, models, training, calibration, evaluation, deployment and debugging.

Show hint

Ask what the model's output is used for, and therefore which property of the prediction matters most besides ordering.

Relevant roles: ML Engineer, Applied Scientist and Data Scientist.

Related free problems