Position Bias in a Click-Trained Feed Ranker

Reported in interviews at Reddit

Problem statement

How would you stop a feed ranker from confusing position with relevance? It learns from clicks and unclicked impressions, but posts near the top get far more clicks. Logs include the user, post, slot, ranker version and click outcome.

Separate relevance from position, estimate examination probabilities and handle unstable weights.

Show hint

What has to happen before a user can click a post, and which of those steps depends on the slot rather than the post?

Relevant roles: ML Engineer, Applied Scientist, Research Scientist and Data Scientist.

Related free problems