Search papers, labs, and topics across Lattice.
The paper introduces StepVAR, a training-free token pruning framework for accelerating visual autoregressive (VAR) models by exploiting the observation that later scales primarily refine textures and exhibit spatial redundancy, while earlier scales determine global structure. StepVAR jointly considers structural and textural importance by using a high-pass filter for texture and PCA for structure, enabling selective token retention. They further introduce a nearest neighbor feature propagation strategy to reconstruct dense feature maps from pruned representations, maintaining valid next-scale prediction.
Achieve significant inference speedups in visual autoregressive models without retraining by pruning redundant tokens based on a novel structure-texture importance criterion.
Visual AutoRegressive (VAR) models based on next-scale prediction enable efficient hierarchical generation, yet the inference cost grows quadratically at high resolutions. We observe that the computationally intensive later scales predominantly refine high-frequency textures and exhibit substantial spatial redundancy, in contrast to earlier scales that determine the global structural layout. Existing pruning methods primarily focus on high-frequency detection for token selection, often overlooking structural coherence and consequently degrading global semantics. To address this limitation, we propose StepVAR, a training-free token pruning framework that accelerates VAR inference by jointly considering structural and textural importance. Specifically, we employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information. This dual-criterion design enables the model to retain tokens critical for both fine-grained fidelity and overall composition. To maintain valid next-scale prediction under sparse tokens, we further introduce a nearest neighbor feature propagation strategy to reconstruct dense feature maps from pruned representations. Extensive experiments on state-of-the-art text-to-image and text-to-video VAR models demonstrate that StepVAR achieves substantial inference speedups while maintaining generation quality. Quantitative and qualitative evaluations consistently show that our method outperforms existing acceleration approaches, validating its effectiveness and general applicability across diverse VAR architectures.
