ML Fundamentals Rapid-Fire: Bias, Serving and Scale
Problem statement
A one-hour ML fundamentals round. The interviewer moves quickly through six topics and wants every answer tied to a concrete situation. Two models from the team's work are used as examples: a regression model that predicts food-delivery time in minutes, and a gradient-boosted classifier that scores card transactions for fraud.
- Bias and variance. For one fixed input $x_0$, the true expected delivery time is 10 minutes and individual deliveries scatter around it with noise variance 4 $\text{min}^2$. You retrain the same model on five independent training sets; its predictions at $x_0$ are 12, 9, 14, 11 and 13 minutes. Treating these five models as the whole distribution of trained models, what is the expected squared error of a prediction against a new delivery at $x_0$? Use the result to explain what bias and variance are and how they relate to underfitting and overfitting.
- Moving along the trade-off. The fraud model (2,000 trees of depth 12, no early stopping) has training AUC 0.995 and AUC 0.86 on a held-out sample from the same three months, split by card so that no card appears in both sets. Which change is most likely to close the gap, and what other levers move a model along the trade-off?
- Parameters vs hyperparameters. In that boosted model, which quantities are parameters and which are hyperparameters, and what sets each of them?
- Batch vs real-time inference. Fraud scores are computed today in a nightly batch job. The business wants every card authorization scored before approval, with a p99 budget of 50 ms. What has to change?
- Training on very large data. The full transaction history is 4 TB, and the largest single machine available has 256 GB of RAM. How do you train on it?
- Feature engineering and the ML lifecycle. What is feature engineering and why does it still matter with modern models? Walk through an end-to-end ML lifecycle from problem framing to iteration, then outline how you would present one project you delivered end to end.
Show hint
For one input, split the expected error into how far the average model is from the truth, how much individual models spread around their average, and what no model can remove.
Relevant roles: ML Engineer, Data Scientist and Applied Scientist.