How Data Collection Choices Shape Downstream Model Performance
Problem statement
How would you measure how training data choices affect a support language model and a ticket-routing classifier? The data mixes ticket transcripts, help articles, forum posts and synthetic answers. Processing includes privacy scrubbing, quality filtering, deduplication, labels and dataset splits, with an internal reference-answer evaluation.
Explain measurable data effects, diagnose filtering losses and quantify evaluation contamination.
Show hint
For each observed pattern, first list what the given facts rule out before naming a cause.
Relevant roles: ML Engineer, LLM Engineer, Applied Scientist and Research Scientist.