Causal and Padding Masks in Batched Decoder Attention
Problem statement
How would you stop a decoder from attending to future tokens or padding? You are implementing attention directly in PyTorch for batches of sequences with different lengths. Training uses next-token prediction and padding at the end of each sequence. Explain causal and padding masks, their tensor shapes, and how to avoid invalid attention rows.
Show hint
For a query at position $i$, list exactly which key positions it may read once both causality and padding are respected.
Relevant roles: LLM Engineer, Deep Learning Engineer, Research Scientist and ML Engineer.
Related free problems
- Batched Causal Self-AttentionML CodingMedium
- Masked Self-AttentionML CodingMedium
- Decoder-Only Transformer Forward Pass from ScratchML CodingHard
- Cross-Document Attention Mask ConstructionML CodingMedium