Search papers, labs, and topics across Lattice.
This paper introduces STORM, a spatial-aware token reduction framework that addresses the performance collapse seen in existing reduction methods applied to Mamba variants. By reformulating token reduction to respect the two-dimensional structure of visual data, STORM preserves grid topology and neighborhood coherence, leading to significant improvements in accuracy. Empirical results show that STORM achieves state-of-the-art pruning accuracy and recovers up to 63.3% in top-1 accuracy for VMamba, while maintaining competitive performance on PlainMamba with only a 1.0% accuracy drop.
STORM recovers up to 63.3% accuracy in visual state space models by enforcing spatial awareness in token reduction, transforming how we approach model efficiency.
Mamba demonstrates strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit a severe performance collapse. We attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism. In this work, we propose STORM, a spatial-aware token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to maintain both grid topology and neighborhood coherence. As a plug-and-play module, STORM equips existing reduction pipelines with explicit spatial awareness without any training. Empirical results demonstrate that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. Notably, STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3\% in top-1 accuracy. Meanwhile, STORM incurs only a 1.0\% accuracy drop on PlainMamba, achieving performance comparable to ViT.
Tsinghua AI