Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Investigating inter-layer residual dynamics across seven open-source model families, the authors challenge the assumption that transformer layers monotonically accumulate persistent features. Using layer Jacobian analysis, they uncover an emergent Transformer Layer Correction Mechanism (TLCM) where adjacent layers operate via a "propose-and-reject" dynamic, systematically suppressing and recalibrating specific subspace updates produced by the preceding layer. This finding reveals that residual streams are heavily populated by transient candidate representations, directly explaining why sparse autoencoders suffer from low feature specificity and why steering interventions require extreme amplification.
Residual streams are dominated by transient feature proposals that adjacent layers immediately cancel out, upending the foundational assumption that transformer representations accumulate monotonically.
Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful features. Such methods are commonly interpreted as identifying features that persist in the residual stream and that subsequent layers build upon. We challenge this view by identifying the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract portions of each other's contributions. TLCM appears in 5 out of 7 major open-source model families and activates across nearly all tokens in diverse texts. We show that TLCM emerges during pretraining, operates most strongly on contextually dependent tokens, and adaptively calibrates its correction strength based on the preceding layer's output. Using the layer Jacobian, we further show that TLCM selectively corrects specific subspaces while reinforcing others, which we interpret through a ``propose-and-reject'' framework in which layers propose candidate features and subsequent layers selectively remove inappropriate ones. This dynamic suggests that the residual stream at any layer contains transient proposals alongside persistent features, helping explain why SAE feature descriptions often have low specificity, why effective model steering requires extreme feature amplification, and why transcoders hold a theoretical advantage over SAEs.