Chapter 1: Classical ML

Decision trees: greedy splits

A service wants to predict which users will cancel. You will score possible questions with numbers, as interviews often ask.

19 minInteractive lessonChapter 1: Classical MLLesson 11 of 21

What you will learn

  • Compute Gini impurity, entropy and information gain
  • Find the best split on one feature by sorting thresholds
  • Explain why depth limits and pruning control overfitting

Key terms

node
A group of users at one point in a tree.
leaf
A final group that gives a prediction.
impurity
A score that measures how mixed a group is.
Gini
An impurity score that uses class proportions.

Prerequisites

Take Preprocessing leakage and train-serve skew first. It comes right before this lesson in the course.

Where this lesson picks up: You asked what to do when a pattern is not a straight line. A chain of yes or no questions is another kind of model.

What comes next

Can several trees reduce the high variance of one deep tree?

Next lesson: Bagging and random forests

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.