Multi-class Cross-Entropy Loss

EasyML Coding

Problem statement

Implement the average categorical cross-entropy loss for a batch of predictions. The function receives predicted_probs of shape (N, C) (each row is a probability distribution over $C$ classes) and one-hot true_labels of the same shape, and returns

$$L = -\frac{1}{N} \sum_{n=1}^{N} \sum_{c=1}^{C} y_{n,c} \log(p_{n,c})$$

as a Python float. For numerical stability, first clip every probability to the range $[\epsilon, 1 - \epsilon]$, where epsilon defaults to 1e-15.

Example

Input:

predicted_probs = np.array([[0.7, 0.2, 0.1], [0.3, 0.6, 0.1]])
true_labels = np.array([[1, 0, 0], [0, 1, 0]])
print(round(compute_cross_entropy_loss(predicted_probs, true_labels), 4))

Output:

0.4338

Reasoning: The probabilities of the correct classes are 0.7 and 0.6, so the loss is $-\frac{1}{2}(\log 0.7 + \log 0.6) \approx 0.4338$.

Show hint

With one-hot labels, which single probability in each row does the loss actually depend on?

Relevant roles: ML Engineer, Deep Learning Engineer, Data Scientist and Applied Scientist.

Related free problems