Chapter 1: Classical ML

Support vector machines: margins and hinge loss

You will be able to compute an SVM's margin and hinge loss by hand and say what its dial $C$ trades. Interviewers ask for exactly this: the widest-gap idea, the hinge loss of one point, and what a large or small $C$ does.

16 minInteractive lessonChapter 1: Classical MLLesson 19 of 26

What you will learn

  • Compute a margin and hinge loss for a soft-margin SVM

Key terms

SVM
support vector machine: it draws a line with a wide gap between two classes
margin
with every dot on its side, the distance from the line to the nearest dots
hinge loss
an SVM's charge for a dot too near the line or on the wrong side
soft margin
an SVM that lets some dots break the gap, for a price
C
a dial you choose: the price of each unit of hinge loss

Prerequisites

Take Two simple baselines: k-nearest neighbours and Naive Bayes first. It comes right before this lesson in the course.

Where this lesson picks up: Last lesson, kNN let the nearest stored customers vote, and Naive Bayes multiplied clue chances. Neither one draws a line between the classes on purpose. What if we choose the line that leaves the most room on both sides?

What comes next

We now have many models, so the next lesson asks: can you trust the scores they output?

Next lesson: ROC, PR curves and AUC

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.