Search papers, labs, and topics across Lattice.
This study investigates the relationship between sparse autoencoder (SAE) features and model behavior by analyzing the geometry of logit changes resulting from feature interventions, rather than focusing solely on feature computation. The researchers introduce Feature-Effect Geometry Analysis (FEGA), revealing that consistent one-dimensional effects are rare across SAE variants, with features categorized as either value-like or pointer-like based on their causal effects. The findings indicate that while features can be interpretable and causally relevant, they often lack stable directional influence, challenging conventional understandings of feature utility in model interpretability.
Few features in sparse autoencoders provide consistent, reusable directions for model behavior, complicating interpretability efforts.
The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior. Features with clear activation descriptions may have weak or unexpected causal effects; steering can vary across prompts or oppose the intended direction; and activation-based feature selection can miss features that produce the desired output change. Prior work has studied feature geometry inside the model, where features are computed. We instead study the geometry of changes in model logits caused by feature interventions. We introduce Feature-Effect Geometry Analysis (FEGA), an unsupervised framework that removes the same active SAE feature across contexts and analyzes the resulting cloud of logit changes. Across SAE variants, consistent one-dimensional effects are rare: few features behave like reusable directions. To interpret this variation, we distinguish value-like features, tied to static information such as factual attributes, from pointer-like features, associated with context-dependent operations. Value-like features more often exhibit structured, low-dimensional effects, although these effects typically span several directions. Pointer-like features, by contrast, predominantly exhibit diffuse effects. Our results show that a feature can be interpretable and causally relevant without providing a stable direction for steering.
