Contrastive Loss (InfoNCE / NT-Xent)

MediumML Coding

Problem statement

Implement the NT-Xent (normalized temperature-scaled cross-entropy) contrastive loss, the InfoNCE-style objective used in SimCLR-style self-supervised learning.

You receive a batch of $2N$ embeddings in which each consecutive pair $(2k, 2k+1)$ holds two augmented views of the same input. The loss pulls the two views of a sample together and pushes views of different samples apart.

Write contrastive_loss(embeddings, temperature) where:

  • embeddings: shape (2N, d); rows (0, 1), (2, 3), (4, 5), ... are positive pairs
  • temperature: a positive float $\tau$

The function should:

  1. L2-normalize every embedding and compute all pairwise cosine similarities.
  2. For each of the $2N$ anchors, treat its paired view as the positive and the other $2N - 2$ embeddings (never the anchor itself) as negatives.
  3. Compute the per-anchor loss

$$\ell_i = -\frac{\text{sim}(z_i, z_{j(i)})}{\tau} + \log \sum_{k \neq i} \exp\left(\frac{\text{sim}(z_i, z_k)}{\tau}\right),$$ where $j(i)$ is the partner of $i$, and return the mean over all $2N$ anchors as a float.

Use the log-sum-exp trick so the function stays finite for very small temperatures.

Example

Input:

embeddings = np.array([[1.0, 0.0], [1.0, 0.0], [0.0, 1.0], [0.0, 1.0]])
contrastive_loss(embeddings, 1.0)

Output:

0.5514

Reasoning: the positive pairs are (0, 1), both [1, 0], and (2, 3), both [0, 1]. For anchor 0 the positive similarity is 1 and the two negatives have similarity 0, so the loss is $-1 + \ln(e + 2) = -1 + 1.5514 = 0.5514$. By symmetry every anchor has the same loss, so the mean is 0.5514.

Show hint

Which classification loss do you get if you treat the partner view as the correct class among all other embeddings in the batch?

Relevant roles: ML Engineer, Research Scientist, Deep Learning Engineer, Applied Scientist and Gen AI Engineer.

Related free problems