Search papers, labs, and topics across Lattice.
This paper investigates the mechanistic origins of attention sinks, a phenomenon where the first token receives disproportionately high attention, in GPT-2-style models. Through structural analysis and causal interventions, the authors identify a circuit involving learned query biases, first-layer MLP transformation of positional encodings, and key projection structure as the primary driver. Surprisingly, they find that each component of this circuit is individually dispensable, suggesting multiple potential circuits can lead to attention sinks across different architectures.
Attention sinks aren't just a quirk of GPT-2; they're a hydra, capable of emerging through multiple distinct circuits, even when you remove the usual suspects.
Transformers commonly exhibit an attention sink: disproportionately high attention to the first position. We study this behavior in GPT-2-style models with learned query biases and absolute positional embeddings. Combining structural analysis with causal interventions, validated across natural-language, mathematical, and code inputs, we find that the sink arises from the interaction among (i) a learned query bias, (ii) the first-layer MLP transformation of the positional encoding, and (iii) structure in the key projection. Crucially, each component we identify is individually dispensable: architectures omitting each of them robustly exhibit sinks. This indicates that attention sinks may arise through distinct circuits across architectures. These findings inform mitigation of sinks, and motivate broader investigation into why sinks emerge.
