Project · 28 steps · Medium

NumPy Multiple Linear Regression GD

Build a from-scratch multiple linear regression trainer in pure NumPy: standardize features, minimize MSE with batch gradient descent and early stopping, compare against the normal equation, and report MAE, RMSE, and R-squared via a reusable model API.

Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.

Part 1: Data Preprocessing

Shuffle and split multi-feature arrays, compute standardization stats, scale features, and build design matrices with a bias column.

  1. shuffle_xy
  2. split_train_val_test
  3. compute_feature_stats
  4. standardize_features
  5. add_bias_column
  6. prepare_design_matrix

Part 2: Prediction, Loss, and Gradients

Implement linear prediction, MSE loss, its analytic gradient, and the closed-form normal-equation solver.

  1. predict_linear
  2. mse_loss
  3. mse_gradient
  4. normal_equation

Part 3: Batch Gradient Descent Training

Initialize weights, take GD steps, track train/val loss, and stop early when validation loss stalls.

  1. initialize_weights
  2. gd_step
  3. epoch_train_val_losses
  4. update_early_stop_state
  5. init_training_state
  6. run_one_epoch
  7. train_batch_gd

Part 4: Evaluation Metrics

Compute MAE, RMSE, and R-squared, plus helpers to package metrics and learning-curve series.

  1. mean_absolute_error
  2. root_mean_squared_error
  3. r_squared
  4. evaluate_regression
  5. learning_curve_data
  6. weights_l2_distance

Part 5: Reusable Model API

Assemble a fit/predict/score model dict that standardizes inputs, trains with GD, stores the normal-equation weights, and compares the two solutions.

  1. create_lr_model
  2. fit_lr_model
  3. predict_lr_model
  4. score_lr_model
  5. compare_with_normal_equation