Hypothesis Test for Tasting Candy Colors
Problem statement
Someone claims they can tell the color of a candy just by tasting it. There are 5 colors total, so a random guesser would get each one right with probability $\frac{1}{5}$.
They run two experiments:
- They taste 3 candies and correctly identify 2 out of 3 colors. Is this convincing evidence that they can actually distinguish colors by taste?
- They taste 100 candies and correctly identify 40 out of 100 colors. What about now?
Formalize this as a hypothesis test and compute the p-value for each case. Use a significance level of $\alpha = 0.05$.
Show hint
What distribution does the number of correct calls follow if the person is purely guessing?
Relevant roles: Data Scientist, Applied Scientist and ML Engineer.