Chapter 1: Classical ML
Features: scaling and missing values
A distance or a penalty adds up numbers from every column, so the column with the biggest units can drown the rest. A gap needs a fill value that one outlier cannot drag. Interviewers ask which models need scaling, and why.
What you will learn
- Pick standardization or min-max scaling and know which models need it
- Impute missing values with a robust statistic
Key terms
- mean $\mu$
- the average: add the values, divide by how many there are
- spread $\sigma$
- the standard deviation: a typical distance of a value from the mean
- outlier
- a value far away from all the others
Prerequisites
Take Features: turning categories into numbers first. It comes right before this lesson in the course.
Where this lesson picks up: Last lesson turned categories into numbers with one-hot columns, ordered codes and hashing. Number columns need care too: they come in different units, and some cells are empty.
What comes next
Each of these fixes learns numbers from data, so which rows should it learn them from?
Next lesson: Preprocessing leakage and train-serve skew
More lessons in this part
- Overfitting and the bias-variance decomposition 21 min
- Learning curves: is it bias or variance, and which fix? 7 min
- Cross-validation and choosing settings 7 min
- Regularization: ridge vs lasso and soft-thresholding 16 min
- Tuning the penalty: cross-validation and the one-standard-error rule 14 min
- Elastic net: when two features carry the same signal 14 min
- Features: turning categories into numbers 14 min
- Preprocessing leakage and train-serve skew 7 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.