Classic machine learning
Support vector machine
Like placing a fence with the widest clear path on both sides, a support vector machine looks for a wide margin between classes. A soft margin allows violations at a cost. Support vectors are the training points that determine the fitted boundary.
Class -1Class +1
How it works
- Scale features using training data and choose a linear boundary or a kernel.
- Score each example with a signed decision function.
- Fit by balancing a wide margin against hinge-loss penalties for margin violations.
- Predict the class from the sign of the fitted score.
$$\min_{w,b}\;\frac12\|w\|_2^2+C\sum_i\max(0,1-y_i(w^Tx_i+b)),\quad y_i\in\{-1,1\}$$
The knobs that matter
- C
- A larger value penalizes margin violations more and weakens regularization.
- Kernel
- A linear or nonlinear similarity function changes available boundary shapes.
- gamma
- For a radial basis kernel, larger gamma makes influence more local.
What interviewers ask
- What is a support vector, and why does the margin matter?
- What does C control in a soft-margin SVM?
- How does the kernel trick let you fit a nonlinear boundary?
- How would you tune C and gamma without leaking test data?
Common mistakes
- Calling the signed decision score a probability without fitting a calibration step.
- Ignoring feature scales or assuming every training point affects the boundary equally.
When to use it
Use a linear SVM for high-dimensional classification or a kernel SVM for moderate datasets needing nonlinear boundaries. Kernel training and prediction can become expensive as the dataset and support-vector count grow.
Implement it (5 problems)
- Hinge Loss for SVMProEasySolve
Build it step by step
- Support Vector Machine from Scratch11 steps, your code runs in the browser
Learn it
Interview questions about support vector machine
- How SVMs and Kernels WorkProMediumSolve