Chapter 1: Classical ML
Elastic net: when two features carry the same signal
You will be able to write one elastic-net update step and say why it makes copied features share their weight. Interviewers ask about both.
What you will learn
- Write the elastic-net gradient step and say why it handles correlated features
Key terms
- copies
- two features that carry the same signal, such as two thermometers in one room
- elastic net
- a fit that adds both a lasso penalty and a ridge penalty
- intercept $b$
- a number added to every prediction
- learning rate $\eta$
- how far each downhill step moves, as a share of the slope
Prerequisites
Take Tuning the penalty: cross-validation and the one-standard-error rule first. It comes right before this lesson in the course.
Where this lesson picks up: Last lesson, you picked the penalty strength $\lambda$ by cross-validation with the one-standard-error rule. But what if two features carry the same signal?
What comes next
Penalties treat weights alike only on one shared scale, so next you turn categories, units and gaps into fair numeric features.
Next lesson: Features: turning categories into numbers
More lessons in this part
- Overfitting and the bias-variance decomposition 21 min
- Learning curves: is it bias or variance, and which fix? 7 min
- Cross-validation and choosing settings 7 min
- Regularization: ridge vs lasso and soft-thresholding 16 min
- Tuning the penalty: cross-validation and the one-standard-error rule 14 min
- Features: turning categories into numbers 14 min
- Features: scaling and missing values 15 min
- Preprocessing leakage and train-serve skew 7 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.