Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the theoretical foundations of Sparse Autoencoders (SAEs) by extending local optimality analyses to understand the properties that optimal dictionaries must satisfy, moving beyond simplistic data-generating models. The authors derive constraints that relate optimal SAE features to their distributions, elucidating behaviors such as hierarchical splitting and dense antipodal features. The findings provide a framework for interpreting SAE outputs and offer principles for the design of future models, enhancing our understanding of how SAEs extract interpretable concepts from complex data.
SAEs reveal interpretable features by adhering to specific optimality constraints, challenging our understanding of how these models operate on complex data.
Sparse Autoencoders (SAEs) have found success parsing neural representations into interpretable concepts, providing a basis for understanding and control. However, what exactly SAEs extract, and, correspondingly, the scientific conclusions we can draw from them, are not obvious. Empirically, the proof is in the pudding: SAEs learn interpretable features. Theoretically, we lack a clear account of what properties a 'concept' must satisfy for an SAE to extract it. There has been extensive identifiability work studying the conditions under which sparse coding recovers ground-truth features; however, these approaches tends to focus on simple data-generating models (e.g. sparse independent features) which poorly approximate the internet-swallowing language-model representations on which SAEs are trained. Here, avoiding data-generating models, we ask simply what properties any dictionary learning optimum must satisfy. Concretely, we extend local optimality analyses (Gribonval & Schnass, 2010) to the nonnegative joint-optimisation problem that vanilla SAEs approximate, and derive constraints relating optimal SAE features to their distributions. We use these constraints to explain a range of observed SAE behaviours - hierarchical splitting & absorption, the structure of residuals, and dense antipodal features - each reflecting how L1+nonnegativity interact with data to structure optimal dictionaries. Finally, we construct a novel large-dictionary convex problem and explore the wide atom-per-datapoint limit. In sum, we hope to tease model assumptions from unexpected observations, letting us learn more from SAEs' successes and provide principles for designing their successors.