Masked Cross-Entropy Loss for Supervised Finetuning
Problem statement
When fine-tuning a language model on prompt-response pairs, compute the training loss only on response tokens. Ignore prompt tokens.
Implement a function masked_ce_loss(logits, targets, mask) that computes the mean cross-entropy loss over only the positions where the mask is True.
Args:
logits: numpy array of shape(seq_len, vocab_size)containing the unnormalized model outputs.targets: numpy array of shape(seq_len,)containing integer token ids in[0, vocab_size).mask: numpy boolean array of shape(seq_len,)where True marks positions that contribute to the loss (response tokens) and False marks ignored positions (prompt tokens).
Return a single Python float: the mean cross-entropy loss averaged over the True positions in the mask. Use a numerically stable log-softmax. If no positions are masked True, return 0.0. Do not round inside the solution.
Example
Input:
logits = [[2.0, 1.0, 0.1], [0.5, 2.5, 0.3], [1.0, 1.0, 1.0]]
targets = [0, 1, 2]
mask = [False, True, True]
Output:
0.6593
Reasoning: Position 0 is masked out (prompt token). For position 1, target=1 with logits [0.5, 2.5, 0.3]: logsumexp = 2.7200, so log-softmax at index 1 = 2.5 - 2.7200 = -0.2200 and the NLL is 0.2200. For position 2, target=2 with uniform logits [1, 1, 1]: log-softmax = log(1/3) = -1.0986, so the NLL is 1.0986. Mean over the two masked positions = (0.2200 + 1.0986) / 2 = 0.6593.
Show hint
Which quantity per position measures how surprised the model is by the correct token, and how do you compute it without overflow?
Relevant roles: LLM Engineer, ML Engineer, Gen AI Engineer, Deep Learning Engineer and Research Scientist.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium