Rebuilding a Job-Post Spam Classifier That Fails in Production

Reported in interviews at Indeed

Problem statement

How would you rebuild a job-post spam classifier that looks strong offline but misses spam in production? Posts need quick decisions, spam is uncommon, and human review capacity is limited. Spammers use generated text and instructions aimed at automated reviewers, while candidate models differ in cost and speed.

Diagnose the evaluation gap, calculate hosted-model capacity, and compare training and serving choices.

Show hint

Before choosing a new model, can you trust the number that says the old one works?

Relevant roles: ML Engineer, Applied Scientist and Data Scientist.

Related free problems