Information Gain of a Decision-Tree Split
Problem statement
A decision-tree node has 10 positive and 10 negative examples. A split sends $(8\text{ pos}, 2\text{ neg})$ left and $(2\text{ pos}, 8\text{ neg})$ right. Define information gain using Shannon entropy, then compute it in bits.
Show hint
How do you measure how mixed a node's labels are, and how would you compare that before and after the split?
Relevant roles: ML Engineer, Data Scientist and Applied Scientist.
Related free problems
- Entropy and Cross-EntropyML CodingEasy