Debugging a Transformer Text Classifier's Failing Tests
Problem statement
How would you debug a text classifier with several failing tests? A PyTorch model fine-tunes a pretrained BERT-style encoder to assign a single class to each customer message. Some bugs are already documented, while others still need diagnosis.
Diagnose the failing tests and show fixes with regression tests.
Show hint
For each failing test, list the causes you can rule out from what the test itself does (mode, padding side, device, precision) before guessing.
Relevant roles: ML Engineer, Deep Learning Engineer and Applied Scientist.
Related free problems
- Debug a Tiny Causal Transformer and Its LossML CodingHard
- Debug a Minimal GPT: Mask, Training Step, Positions, KV CacheLLMs & GenAIHard