Tree Depth, L1 vs L2 vs Weight Decay, and Dropout at Inference

MediumDeep Learning

Reported in interviews at Uber

Problem statement

How would you explain depth and regularization choices in models you trained? A boosted tree ensemble and a small neural network trained with Adam, an L2 penalty and dropout raise questions about overfitting and the difference between training and inference.

Explain tree depth, adaptive regularization and dropout at inference.

Show hint

For each knob, ask what it does to the capacity of a single learner, and how the overall algorithm combines learners or updates weights.

Relevant roles: ML Engineer, Data Scientist and Applied Scientist.

Related free problems