Temperature-Scaled Softmax for Sampling
Problem statement
In language-model text generation, temperature controls how random the next-token distribution is by scaling the logits (raw scores) before the softmax.
Write temperature_sampling(logits, temperature) where:
logitsis a 1D numpy array of raw scores (positive or negative);temperatureis a float that controls how sharp the distribution is.
Return a Python list with the temperature-scaled probability distribution over the vocabulary: $$P(i) = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}.$$
Required behavior:
- $T = 1.0$ gives the standard softmax.
- Low temperatures (close to 0) give sharper, more confident distributions.
- High temperatures give flatter, more uniform distributions.
- If $T \le 0$, return a one-hot distribution with all mass on the highest logit (the first occurrence if there are ties).
The softmax must be numerically stable: large logits must not overflow.
Example
Input:
logits = np.array([1.0, 2.0, 3.0]), temperature = 0.5
Output (rounded to 4 decimals):
[0.0159, 0.1173, 0.8668]
Reasoning: dividing by 0.5 gives $[2, 4, 6]$; subtracting the maximum gives $[-4, -2, 0]$ and $\exp$ of that is $[0.0183, 0.1353, 1.0]$. Dividing by the sum $1.1536$ gives $[0.0159, 0.1173, 0.8668]$, more peaked than the $T = 1$ softmax $[0.09, 0.2447, 0.6652]$.
Show hint
What happens to the gaps between logits when you divide them all by a number smaller or larger than one?
Relevant roles: LLM Engineer, AI Engineer, Gen AI Engineer and ML Engineer.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium