Multi-class Cross-Entropy Loss
Problem statement
Implement the average categorical cross-entropy loss for a batch of predictions. The function receives predicted_probs of shape (N, C) (each row is a probability distribution over $C$ classes) and one-hot true_labels of the same shape, and returns
$$L = -\frac{1}{N} \sum_{n=1}^{N} \sum_{c=1}^{C} y_{n,c} \log(p_{n,c})$$
as a Python float. For numerical stability, first clip every probability to the range $[\epsilon, 1 - \epsilon]$, where epsilon defaults to 1e-15.
Example
Input:
predicted_probs = np.array([[0.7, 0.2, 0.1], [0.3, 0.6, 0.1]])
true_labels = np.array([[1, 0, 0], [0, 1, 0]])
print(round(compute_cross_entropy_loss(predicted_probs, true_labels), 4))
Output:
0.4338
Reasoning: The probabilities of the correct classes are 0.7 and 0.6, so the loss is $-\frac{1}{2}(\log 0.7 + \log 0.6) \approx 0.4338$.
Show hint
With one-hot labels, which single probability in each row does the loss actually depend on?
Relevant roles: ML Engineer, Deep Learning Engineer, Data Scientist and Applied Scientist.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium