Chapter 1: Classical ML

Tuning the penalty: cross-validation and the one-standard-error rule

You will be able to pick $\lambda$ by cross-validation with the one-standard-error rule. Close scores can be luck, and interviewers ask how you would tune a penalty.

14 minInteractive lessonChapter 1: Classical MLLesson 12 of 28

What you will learn

  • Choose the penalty strength by CV with the one-standard-error rule

Key terms

fold
one part of the rows, held out once to test a model
MSE
mean squared error: the average of the squared prediction errors
mean validation error
the average error over the held-out folds
SD
standard deviation: the typical gap between a value and the mean
SE
standard error: how much a mean would wobble with fresh folds

Prerequisites

Take Regularization: ridge vs lasso and soft-thresholding first. It comes right before this lesson in the course.

Where this lesson picks up: Last lesson, ridge and lasso added a penalty whose strength is $\lambda$: a larger $\lambda$ shrinks the weights more. Two lessons ago, cross-validation scored settings on held-out folds. So how strong should $\lambda$ be?

What comes next

Next: what if two features carry the same signal? Lasso may keep just one of them; elastic net makes them share.

Next lesson: Elastic net: when two features carry the same signal

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.