A Boosted Churn Model With a Huge Train-Test Gap
Problem statement
A boosted churn model with many depth-50 trees has training AUC 0.95 and test AUC 0.50. Which change most directly reduces its capacity to overfit?
A. Add more trees. B. Use 3-fold instead of 10-fold cross-validation. C. Log-transform skewed features. D. Reduce the maximum tree depth.
Show hint
What does a model that is nearly perfect on training users and random on new users tell you about its capacity?
Relevant roles: ML Engineer, Applied Scientist and Data Scientist.