Chapter 1: Classical ML
Learning curves: is it bias or variance, and which fix?
You will be able to read a chart of training and validation error against training-set size. You will tell high bias from high variance and choose a fix for each. More of the same data steadies a too-simple model, but it cannot remove its bias, so the plateau stays high. Interviewers test this choice.
What you will learn
- Read a learning curve as high bias or high variance
- Name a fix that targets each
Key terms
- learning curve
- a plot of error against training-set size n
- training-set size $n$
- the number of rows the model learns from
- MSE
- mean squared error: square each miss, then average the squares
Prerequisites
Take Overfitting and the bias-variance decomposition first. It comes right before this lesson in the course.
Where this lesson picks up: You split expected squared error into bias squared, variance and noise, and you saw a perfect training fit fail on new rows. That left a question: when a model does badly, is the cause bias or variance, and what do you change?
What comes next
One validation split gives one lucky or unlucky score, so the next lesson asks how to compare models fairly.
Next lesson: Cross-validation and choosing settings
More lessons in this part
- Overfitting and the bias-variance decomposition 21 min
- Cross-validation and choosing settings 7 min
- Regularization: ridge vs lasso and soft-thresholding 16 min
- Tuning the penalty: cross-validation and the one-standard-error rule 14 min
- Elastic net: when two features carry the same signal 14 min
- Features: turning categories into numbers 14 min
- Features: scaling and missing values 15 min
- Preprocessing leakage and train-serve skew 7 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.