Chapter 1: Classical ML
Decision trees: greedy splits
A service wants to predict which users will cancel. You will score possible questions with numbers, as interviews often ask.
What you will learn
- Compute Gini impurity, entropy and information gain
- Find the best split on one feature by sorting thresholds
- Explain why depth limits and pruning control overfitting
Key terms
- node
- A group of users at one point in a tree.
- leaf
- A final group that gives a prediction.
- impurity
- A score that measures how mixed a group is.
- Gini
- An impurity score that uses class proportions.
Prerequisites
Take Preprocessing leakage and train-serve skew first. It comes right before this lesson in the course.
Where this lesson picks up: You asked what to do when a pattern is not a straight line. A chain of yes or no questions is another kind of model.
What comes next
Can several trees reduce the high variance of one deep tree?
Next lesson: Bagging and random forests
More lessons in this part
- Bagging and random forests 25 min
- Gradient boosting 21 min
- Two simple baselines: k-nearest neighbours and Naive Bayes 17 min
- Support vector machines: margins and hinge loss 16 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.