Tree Depth, L1 vs L2 vs Weight Decay, and Dropout at Inference
Problem statement
How would you explain depth and regularization choices in models you trained? A boosted tree ensemble and a small neural network trained with Adam, an L2 penalty and dropout raise questions about overfitting and the difference between training and inference.
Explain tree depth, adaptive regularization and dropout at inference.
Show hint
For each knob, ask what it does to the capacity of a single learner, and how the overall algorithm combines learners or updates weights.
Relevant roles: ML Engineer, Data Scientist and Applied Scientist.