How Many Features Does Mutual-Information Filtering Keep?
Problem statement
How many features would a mutual-information filter keep? There are 50 individually informative features, one exact copy of each, and 900 features independent of the label. Score each feature against the label separately using exact mutual information. Keep scores above a positive threshold that is below every informative feature's score.
Show hint
Does a mutual-information filter ever look at two features at the same time?
Relevant roles: ML Engineer, Data Scientist and Applied Scientist.