Project · 27 steps · Hard
NumPy Text Classifier from Scratch
Build a complete spam/sentiment text classifier in pure NumPy: clean and tokenize text, extract fixed-vocabulary TF-IDF features, train L2-regularized logistic regression with batch gradient descent, tune the decision threshold on validation data, and deliver a predict(text) API with false-positive/false-negative error analysis.
Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.
Part 1: Text Preprocessing
Clean raw strings and tokenize them into word lists ready for feature extraction.
clean_texttokenizetokenize_corpus
Part 2: Vocabulary and Bag-of-Words
Split data, build a fixed vocabulary from training documents only, and convert tokenized text into bag-of-words count vectors.
split_train_val_test_indicescount_word_frequenciesbuild_vocabularytokens_to_bowcorpus_to_bow_matrix
Part 3: TF-IDF Features
Compute inverse document frequencies from the training BoW matrix and transform count vectors into TF-IDF feature matrices.
compute_document_frequenciescompute_idftransform_tfidffit_tfidf
Part 4: Logistic Regression Core
Implement sigmoid, probability prediction, binary cross-entropy with L2 penalty, and gradient computation.
sigmoidlogistic_predict_probabinary_cross_entropylogistic_gradients
Part 5: Training and Threshold Tuning
Train weights with full-batch gradient descent and select the decision threshold that maximizes F1 on a validation set.
initialize_logistic_paramsgradient_descent_steptrain_logistic_regressionpredict_labelsconfusion_countsmetrics_from_countstune_decision_threshold
Part 6: Evaluation and Predict API
Compute confusion-matrix metrics, expose an end-to-end predict(text) API, and collect false positives vs false negatives for error analysis.
evaluate_predictionsvectorize_textspredict_textcollect_prediction_errors