Detecting and Handling Data Shift in Deployed Models
Problem statement
How would you keep a loan fraud model useful after launch? Confirmed fraud can take months to arrive, and you only observe outcomes for approved loans. The team sees input distributions moving and later measures a drop in performance. Explain how to detect shifts, choose a response and account for delayed labels.
Show hint
Separate changes in the inputs, in the base rate and in the input-to-label relationship.
Relevant roles: ML Engineer, MLOps Engineer, Applied Scientist and Data Scientist.