Chapter 1: Classical ML

Elastic net: when two features carry the same signal

You will be able to write one elastic-net update step and say why it makes copied features share their weight. Interviewers ask about both.

14 minInteractive lessonChapter 1: Classical MLLesson 13 of 28

What you will learn

  • Write the elastic-net gradient step and say why it handles correlated features

Key terms

copies
two features that carry the same signal, such as two thermometers in one room
elastic net
a fit that adds both a lasso penalty and a ridge penalty
intercept $b$
a number added to every prediction
learning rate $\eta$
how far each downhill step moves, as a share of the slope

Prerequisites

Take Tuning the penalty: cross-validation and the one-standard-error rule first. It comes right before this lesson in the course.

Where this lesson picks up: Last lesson, you picked the penalty strength $\lambda$ by cross-validation with the one-standard-error rule. But what if two features carry the same signal?

What comes next

Penalties treat weights alike only on one shared scale, so next you turn categories, units and gaps into fair numeric features.

Next lesson: Features: turning categories into numbers

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.