Search papers, labs, and topics across Lattice.
This study investigates the effects of pruning on the robustness of sparse autoencoders (SAEs) used in large language models (LLMs), revealing that traditional magnitude pruning distorts learned representations due to its neglect of activation geometry. By introducing an activation-aware pruning approach, the authors demonstrate that methods like Wanda and SparseGPT can maintain SAE functionality more effectively by controlling perturbation energy. Additionally, they identify a structural vulnerability where middle layers are more sensitive to pruning, leading to a proposed layer-wise sparsity allocation strategy that reduces perplexity while maintaining average pruning sparsity across various model architectures.
Pruning can significantly undermine sparse autoencoder performance, but activation-aware methods offer a robust alternative that preserves interpretability in LLMs.
Sparse autoencoders (SAEs) are widely used to interpret the internal representations of large language models (LLMs), yet their reliability under post-hoc model compression remains poorly understood. We present a systematic study of how pruning affects SAE behavior and theoretically show that, for a fixed SAE, its impact is governed by perturbation energy, a covariance-weighted norm. This perspective exposes a key limitation of magnitude pruning: by ignoring activation geometry, it distorts the learned representation space and degrades SAE functionality. Activation-aware methods such as Wanda and SparseGPT, in contrast, implicitly control perturbation energy and are therefore substantially more robust at preserving SAE behavior. We further reveal a consistent structural vulnerability across all pruning methods: middle layers are significantly more sensitive to pruning than early or late layers. Guided by this insight, we propose a layer-wise sparsity allocation strategy, achieving lower perplexity under the same average pruning sparsity. Experiments across four model architectures validate our theoretical findings. Code is publicly available at https://github.com/osu-srml/sae-robustness-under-pruning/tree/main.
