ML Fundamentals Round: Cross-Entropy, Dropout, Normalization, Adam and PPO
Problem statement
How would you reason through a few common model training mistakes? We will start with a classifier trained on annotators' soft labels, then examine a ranking model whose dropout changes score scale at serving. Finally, we will consider normalization in a language model that generates tokens incrementally.
Calculate the reducible loss, fix dropout scaling and explain the normalization choice.
Show hint
For each question, ask what the definition implies about a concrete quantity: a loss floor, an activation scale, the axis statistics are taken over.
Relevant roles: ML Engineer, Deep Learning Engineer, Research Scientist and Applied Scientist.
Related free problems
- AdamW Optimizer StepML CodingEasy