Temperature-Scaled Softmax for Sampling

EasyML Coding

Problem statement

In language-model text generation, temperature controls how random the next-token distribution is by scaling the logits (raw scores) before the softmax.

Write temperature_sampling(logits, temperature) where:

  • logits is a 1D numpy array of raw scores (positive or negative);
  • temperature is a float that controls how sharp the distribution is.

Return a Python list with the temperature-scaled probability distribution over the vocabulary: $$P(i) = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}.$$

Required behavior:

  • $T = 1.0$ gives the standard softmax.
  • Low temperatures (close to 0) give sharper, more confident distributions.
  • High temperatures give flatter, more uniform distributions.
  • If $T \le 0$, return a one-hot distribution with all mass on the highest logit (the first occurrence if there are ties).

The softmax must be numerically stable: large logits must not overflow.

Example

Input:

logits = np.array([1.0, 2.0, 3.0]), temperature = 0.5

Output (rounded to 4 decimals):

[0.0159, 0.1173, 0.8668]

Reasoning: dividing by 0.5 gives $[2, 4, 6]$; subtracting the maximum gives $[-4, -2, 0]$ and $\exp$ of that is $[0.0183, 0.1353, 1.0]$. Dividing by the sum $1.1536$ gives $[0.0159, 0.1173, 0.8668]$, more peaked than the $T = 1$ softmax $[0.09, 0.2447, 0.6652]$.

Show hint

What happens to the gaps between logits when you divide them all by a number smaller or larger than one?

Relevant roles: LLM Engineer, AI Engineer, Gen AI Engineer and ML Engineer.

Related free problems