Batched Attention Score Computation for Multiple Heads
Problem statement
Implement a function batched_attention_scores(Q, K) that computes scaled dot-product attention scores for a batched, multi-head input.
Given query tensor Q and key tensor K, both with shape (batch, num_heads, seq_len, head_dim), your function should:
- Compute the unnormalized attention scores by matrix-multiplying
Qwith the transpose ofKalong the last two dimensions. The result has shape(batch, num_heads, seq_len, seq_len). - Scale the resulting scores by dividing by
sqrt(head_dim). - Return the scaled scores as a nested Python list (via
.tolist()).
Parameters:
Q: numpy array of shape(batch, num_heads, seq_len, head_dim)K: numpy array of shape(batch, num_heads, seq_len, head_dim)
Returns:
- A nested list representing a numpy array of shape
(batch, num_heads, seq_len, seq_len).
The transpose must be applied only to the last two dimensions of K, leaving the batch and head dimensions intact, so that each head in each batch computes its own independent score matrix.
Example
Input:
Q = [[[[1, 0], [0, 1]]]] # shape (1, 1, 2, 2)
K = [[[[1, 0], [0, 1]]]] # shape (1, 1, 2, 2)
Output:
[[[[0.7071, 0.0], [0.0, 0.7071]]]]
Reasoning: Q @ K^T is the 2x2 identity. Dividing by sqrt(head_dim) = sqrt(2) gives 0.7071 on the diagonal and 0 elsewhere. The batch and head dimensions are preserved, so the result has shape (1, 1, 2, 2).
Show hint
Which axes of a 4D key tensor actually take part in the per-head matrix product?
Relevant roles: ML Engineer, LLM Engineer, Deep Learning Engineer and AI Engineer.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium