Metrics, L1 vs L2 and an Ablation Study for a Ticket-Urgency NLP Pipeline
Problem statement
How would you detect urgent support tickets and decide which pipeline components help? Urgent tickets are rare and the on-call desk has limited review capacity. The pipeline uses text processing, retrieval, reranking and an LLM, while a compact word-feature model serves as a fallback. Choose launch metrics and regularization, then design a controlled ablation study.
Show hint
For each question, ask what the people consuming the output actually see: a queue of top-scored tickets, a memory budget, or a release decision.
Relevant roles: ML Engineer, Applied Scientist and Data Scientist.