Project · 27 steps · Hard

NumPy Text Classifier from Scratch

Build a complete spam/sentiment text classifier in pure NumPy: clean and tokenize text, extract fixed-vocabulary TF-IDF features, train L2-regularized logistic regression with batch gradient descent, tune the decision threshold on validation data, and deliver a predict(text) API with false-positive/false-negative error analysis.

Each step is one function with tests. Your code runs in the browser, and every step builds on the ones before it.

Part 1: Text Preprocessing

Clean raw strings and tokenize them into word lists ready for feature extraction.

  1. clean_text
  2. tokenize
  3. tokenize_corpus

Part 2: Vocabulary and Bag-of-Words

Split data, build a fixed vocabulary from training documents only, and convert tokenized text into bag-of-words count vectors.

  1. split_train_val_test_indices
  2. count_word_frequencies
  3. build_vocabulary
  4. tokens_to_bow
  5. corpus_to_bow_matrix

Part 3: TF-IDF Features

Compute inverse document frequencies from the training BoW matrix and transform count vectors into TF-IDF feature matrices.

  1. compute_document_frequencies
  2. compute_idf
  3. transform_tfidf
  4. fit_tfidf

Part 4: Logistic Regression Core

Implement sigmoid, probability prediction, binary cross-entropy with L2 penalty, and gradient computation.

  1. sigmoid
  2. logistic_predict_proba
  3. binary_cross_entropy
  4. logistic_gradients

Part 5: Training and Threshold Tuning

Train weights with full-batch gradient descent and select the decision threshold that maximizes F1 on a validation set.

  1. initialize_logistic_params
  2. gradient_descent_step
  3. train_logistic_regression
  4. predict_labels
  5. confusion_counts
  6. metrics_from_counts
  7. tune_decision_threshold

Part 6: Evaluation and Predict API

Compute confusion-matrix metrics, expose an end-to-end predict(text) API, and collect false positives vs false negatives for error analysis.

  1. evaluate_predictions
  2. vectorize_texts
  3. predict_text
  4. collect_prediction_errors