Chapter 1: Classical ML

Training logistic regression: log loss, its gradient, and why not squared error

A model can learn only with a score for each guess and a direction for each weight. You will be able to score a predicted chance and find which way each weight must move. You will also see why squared error trains this model badly, a common interview question.

7 minInteractive lessonChapter 1: Classical MLLesson 5 of 26

What you will learn

  • Compute log loss and the gradient (p - y) x for one example
  • Explain why squared error is a poor loss for a sigmoid output

Key terms

log loss
a score for one guess: minus ln of the chance it gave to what happened
ln
the natural log, the log with base e
gradient
the slope of the loss for a weight, which shows which way to move it
likelihood
the chance the model gave to all the outcomes that really happened

Prerequisites

Take From a line to a probability: the sigmoid first. It comes right before this lesson in the course.

Where this lesson picks up: Last time, the sigmoid turned a line's score into a chance between 0 and 1. The model now says 0.7 for a customer who stayed, so how do we score that guess, and which way should the weight move?

What comes next

The model says 0.3 for a customer: do we act?

Next lesson: Thresholds, precision and recall: the cut-off is a product decision

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.