Binary Cross-Entropy Loss
Problem statement
Implement the binary cross-entropy (BCE) loss, the standard loss for binary classification.
Given a list of true labels (each 0 or 1) and a list of predicted probabilities in $[0, 1]$, return the mean BCE over all samples as a float:
$$\text{BCE} = -\frac{1}{N} \sum_{i=1}^{N} \big[y_i \log p_i + (1 - y_i) \log(1 - p_i)\big].$$
Predictions of exactly 0 or 1 would make a logarithm infinite, so first clip every prediction to $[\epsilon, 1 - \epsilon]$ using the epsilon argument (default 1e-15).
Write binary_cross_entropy(y_true, y_pred, epsilon=1e-15). Use natural logarithms and return the full float, without rounding. Assume the lists are nonempty and have equal length, and $0 < \epsilon < 0.5$.
Example
Input:
y_true = [1, 0, 1, 0], y_pred = [0.9, 0.1, 0.8, 0.2]
Output:
0.1643
Reasoning: For each sample, we compute: -[y log(p) + (1-y) log(1-p)]. Sample 0: -(1log(0.9) + 0log(0.1)) = 0.1054. Sample 1: -(0log(0.1) + 1log(0.9)) = 0.1054. Sample 2: -log(0.8) = 0.2231. Sample 3: -log(0.8) = 0.2231. Mean = (0.1054 + 0.1054 + 0.2231 + 0.2231) / 4 = 0.1643
Show hint
Which probability model for a 0/1 outcome gives this loss as its negative log-likelihood?
Relevant roles: ML Engineer, Data Scientist, Applied Scientist and Deep Learning Engineer.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium