Masked Cross-Entropy Loss for Supervised Finetuning

EasyML Coding

Problem statement

When fine-tuning a language model on prompt-response pairs, compute the training loss only on response tokens. Ignore prompt tokens.

Implement a function masked_ce_loss(logits, targets, mask) that computes the mean cross-entropy loss over only the positions where the mask is True.

Args:

  • logits: numpy array of shape (seq_len, vocab_size) containing the unnormalized model outputs.
  • targets: numpy array of shape (seq_len,) containing integer token ids in [0, vocab_size).
  • mask: numpy boolean array of shape (seq_len,) where True marks positions that contribute to the loss (response tokens) and False marks ignored positions (prompt tokens).

Return a single Python float: the mean cross-entropy loss averaged over the True positions in the mask. Use a numerically stable log-softmax. If no positions are masked True, return 0.0. Do not round inside the solution.

Example

Input:

logits = [[2.0, 1.0, 0.1], [0.5, 2.5, 0.3], [1.0, 1.0, 1.0]]
targets = [0, 1, 2]
mask = [False, True, True]

Output:

0.6593

Reasoning: Position 0 is masked out (prompt token). For position 1, target=1 with logits [0.5, 2.5, 0.3]: logsumexp = 2.7200, so log-softmax at index 1 = 2.5 - 2.7200 = -0.2200 and the NLL is 0.2200. For position 2, target=2 with uniform logits [1, 1, 1]: log-softmax = log(1/3) = -1.0986, so the NLL is 1.0986. Mean over the two masked positions = (0.2200 + 1.0986) / 2 = 0.6593.

Show hint

Which quantity per position measures how surprised the model is by the correct token, and how do you compute it without overflow?

Relevant roles: LLM Engineer, ML Engineer, Gen AI Engineer, Deep Learning Engineer and Research Scientist.

Related free problems