ML Fundamentals: Metrics, Gradients, Ensembles and a Forward Pass

EasyClassical ML

Reported in interviews at Pinterest

Problem statement

A timed online assessment opens with short ML fundamentals questions. They all refer to the small scenarios below.

Classifier. A model that flags policy-violating posts is evaluated on 10,000 labeled posts (violating = positive). Its confusion matrix: 160 true positives, 40 false positives, 240 false negatives and 9,560 true negatives.

Gradient descent. A loss over two weights is $L(w_1, w_2) = (w_1 - 3)^2 + 2 w_2^2$. Training starts at $(w_1, w_2) = (1, 1)$ and uses plain gradient descent with learning rate $\eta = 0.1$.

Trees. On a tabular dataset, a single fully grown decision tree scores 99% accuracy on training data and 81% on validation data. A depth-1 tree (a stump) scores 70% on training and 69% on validation.

Small network. A feed-forward network takes $x = (2, -1)$. The hidden layer has three ReLU units with $$W_1 = \begin{pmatrix} 1 & 2 \\ 3 & 1 \\ -1 & 1 \end{pmatrix}, \qquad b_1 = (0.5, -1, 1).$$ The output unit is a sigmoid with weights $w_2 = (2, -0.5, 3)$ and bias $b_2 = 0.5$.

  1. Compute the classifier's precision and its recall, and say in words what each one measures.
  2. The team appends 5,000 more benign posts to the evaluation set, and the model correctly predicts all of them as negative. Which metrics change and which do not?
  3. What does the gradient of $L$ represent? Take one gradient descent step from $(1, 1)$ and report the loss at the new point.
  4. Which ensemble method and base learner best address the full tree's problem, and why? Contrast bagging and boosting on bias, variance and compute.
  5. Compute the network's output probability by hand.
Show hint

Gradient descent subtracts the learning rate times the gradient; compute the partial derivatives at the starting point first.

Relevant roles: ML Engineer, Data Scientist and Applied Scientist.

Related free problems