Designing a MapReduce Pipeline for Daily User Features
Problem statement
How would you build daily user features with MapReduce? A very large event stream feeds historical training data and an online store that needs the latest day. The nightly job has a tight completion window and must stay correct through retries and failures.
Design storage, safe output commits and an efficient, measurable pipeline.
Show hint
Follow the bytes: what each stage reads, what it sends over the network, what it writes, and who reads the output afterwards and how.
Relevant roles: ML Engineer and MLOps Engineer.