Project · 11 steps · Medium

Support Vector Machine from Scratch

Build a linear Support Vector Machine end-to-end using only basic array operations, implementing the hinge loss, regularized objective, and gradient-based training loop. By the end you'll have a working binary classifier you can train, predict with, and evaluate on real data.

Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.

Part 1: Data Preparation

Prepare the input features so the model can train stably.

  1. standardize_features

Part 2: Model Setup and Forward Pass

Initialize parameters and compute scores and predictions from inputs.

  1. initialize_parameters
  2. compute_scores
  3. predict_from_scores

Part 3: Loss and Objective

Define the per-example hinge loss and the full regularized training objective.

  1. hinge_loss_example
  2. svm_objective

Part 4: Gradients and Parameter Updates

Derive gradients of the objective and apply single parameter updates.

  1. compute_gradients
  2. apply_update

Part 5: Training Loop

Repeat updates over epochs to fit the model to the data.

  1. train_svm

Part 6: Prediction and Evaluation

Use the trained model to classify new data and score its accuracy.

  1. predict_labels
  2. accuracy_score