Project · 28 steps · Medium
NumPy Multiple Linear Regression GD
Build a from-scratch multiple linear regression trainer in pure NumPy: standardize features, minimize MSE with batch gradient descent and early stopping, compare against the normal equation, and report MAE, RMSE, and R-squared via a reusable model API.
Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.
Part 1: Data Preprocessing
Shuffle and split multi-feature arrays, compute standardization stats, scale features, and build design matrices with a bias column.
shuffle_xysplit_train_val_testcompute_feature_statsstandardize_featuresadd_bias_columnprepare_design_matrix
Part 2: Prediction, Loss, and Gradients
Implement linear prediction, MSE loss, its analytic gradient, and the closed-form normal-equation solver.
predict_linearmse_lossmse_gradientnormal_equation
Part 3: Batch Gradient Descent Training
Initialize weights, take GD steps, track train/val loss, and stop early when validation loss stalls.
initialize_weightsgd_stepepoch_train_val_lossesupdate_early_stop_stateinit_training_staterun_one_epochtrain_batch_gd
Part 4: Evaluation Metrics
Compute MAE, RMSE, and R-squared, plus helpers to package metrics and learning-curve series.
mean_absolute_errorroot_mean_squared_errorr_squaredevaluate_regressionlearning_curve_dataweights_l2_distance
Part 5: Reusable Model API
Assemble a fit/predict/score model dict that standardizes inputs, trains with GD, stores the normal-equation weights, and compares the two solutions.
create_lr_modelfit_lr_modelpredict_lr_modelscore_lr_modelcompare_with_normal_equation