ML Fundamentals: Metrics, Gradients, Ensembles and a Forward Pass
Problem statement
A timed online assessment opens with short ML fundamentals questions. They all refer to the small scenarios below.
Classifier. A model that flags policy-violating posts is evaluated on 10,000 labeled posts (violating = positive). Its confusion matrix: 160 true positives, 40 false positives, 240 false negatives and 9,560 true negatives.
Gradient descent. A loss over two weights is $L(w_1, w_2) = (w_1 - 3)^2 + 2 w_2^2$. Training starts at $(w_1, w_2) = (1, 1)$ and uses plain gradient descent with learning rate $\eta = 0.1$.
Trees. On a tabular dataset, a single fully grown decision tree scores 99% accuracy on training data and 81% on validation data. A depth-1 tree (a stump) scores 70% on training and 69% on validation.
Small network. A feed-forward network takes $x = (2, -1)$. The hidden layer has three ReLU units with $$W_1 = \begin{pmatrix} 1 & 2 \\ 3 & 1 \\ -1 & 1 \end{pmatrix}, \qquad b_1 = (0.5, -1, 1).$$ The output unit is a sigmoid with weights $w_2 = (2, -0.5, 3)$ and bias $b_2 = 0.5$.
- Compute the classifier's precision and its recall, and say in words what each one measures.
- The team appends 5,000 more benign posts to the evaluation set, and the model correctly predicts all of them as negative. Which metrics change and which do not?
- What does the gradient of $L$ represent? Take one gradient descent step from $(1, 1)$ and report the loss at the new point.
- Which ensemble method and base learner best address the full tree's problem, and why? Contrast bagging and boosting on bias, variance and compute.
- Compute the network's output probability by hand.
Show hint
Gradient descent subtracts the learning rate times the gradient; compute the partial derivatives at the starting point first.
Relevant roles: ML Engineer, Data Scientist and Applied Scientist.