Project · 15 steps · Medium
Random Forest from Scratch
Build a complete Random Forest classifier from the ground up, starting with the impurity math behind a single decision tree and ending with a bagged ensemble that votes on predictions. Along the way you'll implement splitting, recursion, bootstrapping, and feature subsampling, the core ideas behind one of the most reliable classical ML models.
Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.
Part 1: Impurity and Splitting Primitives
Build the low-level tools a decision tree needs: measuring impurity, partitioning data on a feature threshold, and scoring candidate splits.
impuritysplit_datasetsplit_score
Part 2: Growing a Decision Tree
Combine the primitives into a recursive tree builder with best-split selection, stopping criteria, and leaf prediction.
best_splitshould_stopleaf_predictionbuild_tree
Part 3: Predicting with a Single Tree
Traverse a fitted tree to predict single examples and full datasets.
predict_example_treepredict_tree
Part 4: Randomness for the Ensemble
Add bootstrap sampling of rows and random feature subsets so that individual trees become decorrelated.
bootstrap_samplefeature_subset
Part 5: Training and Predicting with the Forest
Grow many trees on randomized data, aggregate their votes, produce forest-level predictions, and evaluate accuracy.
train_forestcombine_predictionspredict_forestaccuracy