Chapter 1: Classical ML
Overfitting and the bias-variance decomposition
A model can fit every training row perfectly and still fail on new rows. You will be able to split the average squared error on a new sale into a steady miss, a jump between training sets, and noise. You will also be able to explain why a perfect training fit can mean nothing, a common interview question.
What you will learn
- State the bias-variance decomposition of squared error
Key terms
- predictor
- An input column that the model uses to predict the price.
- intercept
- A number added to every prediction.
- coefficient
- An adjustable number in the model. The fit sets its value.
- independent rows
- Rows where no row is a combination of the other rows.
Prerequisites
Take Honest splits and leakage: random, group and time-based first. It comes right before this lesson in the course.
Where this lesson picks up: You kept the evaluation rows out of training, and the honest held-out score came out lower than the training score. That left a question: why is the honest score lower?
What comes next
Next, you will find which of the three parts hurts your model, and what to change.
Next lesson: Learning curves: is it bias or variance, and which fix?
More lessons in this part
- Learning curves: is it bias or variance, and which fix? 7 min
- Cross-validation and choosing settings 7 min
- Regularization: ridge vs lasso and soft-thresholding 16 min
- Tuning the penalty: cross-validation and the one-standard-error rule 14 min
- Elastic net: when two features carry the same signal 14 min
- Features: turning categories into numbers 14 min
- Features: scaling and missing values 15 min
- Preprocessing leakage and train-serve skew 7 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.