Chapter 1: Classical ML
Support vector machines: margins and hinge loss
You will be able to compute an SVM's margin and hinge loss by hand and say what its dial $C$ trades. Interviewers ask for exactly this: the widest-gap idea, the hinge loss of one point, and what a large or small $C$ does.
What you will learn
- Compute a margin and hinge loss for a soft-margin SVM
Key terms
- SVM
- support vector machine: it draws a line with a wide gap between two classes
- margin
- with every dot on its side, the distance from the line to the nearest dots
- hinge loss
- an SVM's charge for a dot too near the line or on the wrong side
- soft margin
- an SVM that lets some dots break the gap, for a price
- C
- a dial you choose: the price of each unit of hinge loss
Prerequisites
Take Two simple baselines: k-nearest neighbours and Naive Bayes first. It comes right before this lesson in the course.
Where this lesson picks up: Last lesson, kNN let the nearest stored customers vote, and Naive Bayes multiplied clue chances. Neither one draws a line between the classes on purpose. What if we choose the line that leaves the most room on both sides?
What comes next
We now have many models, so the next lesson asks: can you trust the scores they output?
Next lesson: ROC, PR curves and AUC
More lessons in this part
- Decision trees: greedy splits 19 min
- Bagging and random forests 25 min
- Gradient boosting 21 min
- Two simple baselines: k-nearest neighbours and Naive Bayes 17 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.