Classic machine learning

Support vector machine

Like placing a fence with the widest clear path on both sides, a support vector machine looks for a wide margin between classes. A soft margin allows violations at a cost. Support vectors are the training points that determine the fitted boundary.

-3-2-10123-2-1012Feature 1Feature 2MarginBoundaryMargin
Class -1Class +1
For score f(x) = x1, the boundary is x1 = 0; support vectors at (-1, 0) and (1, 0) touch margins two units apart.

How it works

  1. Scale features using training data and choose a linear boundary or a kernel.
  2. Score each example with a signed decision function.
  3. Fit by balancing a wide margin against hinge-loss penalties for margin violations.
  4. Predict the class from the sign of the fitted score.

$$\min_{w,b}\;\frac12\|w\|_2^2+C\sum_i\max(0,1-y_i(w^Tx_i+b)),\quad y_i\in\{-1,1\}$$

The knobs that matter

C
A larger value penalizes margin violations more and weakens regularization.
Kernel
A linear or nonlinear similarity function changes available boundary shapes.
gamma
For a radial basis kernel, larger gamma makes influence more local.

What interviewers ask

  • What is a support vector, and why does the margin matter?
  • What does C control in a soft-margin SVM?
  • How does the kernel trick let you fit a nonlinear boundary?
  • How would you tune C and gamma without leaking test data?

Common mistakes

  • Calling the signed decision score a probability without fitting a calibration step.
  • Ignoring feature scales or assuming every training point affects the boundary equally.

When to use it

Use a linear SVM for high-dimensional classification or a kernel SVM for moderate datasets needing nonlinear boundaries. Kernel training and prediction can become expensive as the dataset and support-vector count grow.

Implement it (5 problems)

Build it step by step

Learn it

Interview questions about support vector machine