Hila Ofek

Papers on Lattice

Total citations

Topics

h-index

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Interpretability & Mechanistic Interp (1)Natural Language Processing (1)

Frequent co-authors

Yuval Ran-milo (1)Shahar Mendel (1)

Papers (1)

Apr 16, 2026

A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation

Attention sinks aren't just a quirk of GPT-2; they're a hydra, capable of emerging through multiple distinct circuits, even when you remove the usual suspects.

Yuval Ran-milo, Hila Ofek, Shahar Mendel

Architecture Design (Transformers, SSMs, MoE)Interpretability & Mechanistic Interp Natural Language Processing

Search

Hila Ofek

Research focus

Frequent co-authors

Papers (1)