Chapter 1: Classical ML
Two simple baselines: k-nearest neighbours and Naive Bayes
You will be able to make one prediction by hand with two simple classic models and name how each one fails. One asks the nearest look-alikes, and one multiplies clue chances. Both are quick baselines: first models that a better model must beat. Interviewers ask how each one decides and where it breaks.
What you will learn
- Predict with kNN and explain why distances lose meaning in high dimensions
- Compute a Naive Bayes posterior and state its independence assumption
Key terms
- kNN
- k-nearest neighbours: the k most similar stored cases vote on a guess
- Naive Bayes
- it multiplies a start chance by one chance per clue, class by class
- prior
- a class's share before you look at any clue
- posterior
- a class's chance after you use the clues
- baseline
- a simple first model that a better model must beat
Prerequisites
Take Gradient boosting first. It comes right before this lesson in the course.
Where this lesson picks up: You built gradient boosting, where each small tree fixes the errors that the trees before it left. That closed the tree family and left a question: what other ideas did classic machine learning try?
What comes next
Next, you will meet a model that draws the dividing line with the most room on both sides: the support vector machine.
Next lesson: Support vector machines: margins and hinge loss
More lessons in this part
- Decision trees: greedy splits 19 min
- Bagging and random forests 25 min
- Gradient boosting 21 min
- Support vector machines: margins and hinge loss 16 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.