Chapter 1: Classical ML

Two simple baselines: k-nearest neighbours and Naive Bayes

You will be able to make one prediction by hand with two simple classic models and name how each one fails. One asks the nearest look-alikes, and one multiplies clue chances. Both are quick baselines: first models that a better model must beat. Interviewers ask how each one decides and where it breaks.

17 minInteractive lessonChapter 1: Classical MLLesson 18 of 26

What you will learn

  • Predict with kNN and explain why distances lose meaning in high dimensions
  • Compute a Naive Bayes posterior and state its independence assumption

Key terms

kNN
k-nearest neighbours: the k most similar stored cases vote on a guess
Naive Bayes
it multiplies a start chance by one chance per clue, class by class
prior
a class's share before you look at any clue
posterior
a class's chance after you use the clues
baseline
a simple first model that a better model must beat

Prerequisites

Take Gradient boosting first. It comes right before this lesson in the course.

Where this lesson picks up: You built gradient boosting, where each small tree fixes the errors that the trees before it left. That closed the tree family and left a question: what other ideas did classic machine learning try?

What comes next

Next, you will meet a model that draws the dividing line with the most room on both sides: the support vector machine.

Next lesson: Support vector machines: margins and hinge loss

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.