Search papers, labs, and topics across Lattice.
The paper introduces ToProVAR, a novel optimization framework for visual autoregressive (VAR) models that addresses the efficiency bottleneck in later generation stages. ToProVAR leverages attention entropy to characterize semantic projections across different dimensions, enabling precise identification of parameter dynamics under varying token granularity levels, semantic scopes, and generation scales. By uncovering sparsity patterns along token, layer, and scale dimensions, ToProVAR achieves up to 3.4x acceleration with minimal quality loss compared to existing methods like FastVAR and SkipVAR.
Attention entropy reveals exploitable sparsity in VAR models, enabling 3.4x faster image generation without sacrificing quality.
Visual Autoregressive(VAR) models enhance generation quality but face a critical efficiency bottleneck in later stages. In this paper, we present a novel optimization framework for VAR models that fundamentally differs from prior approaches such as FastVAR and SkipVAR. Instead of relying on heuristic skipping strategies, our method leverages attention entropy to characterize the semantic projections across different dimensions of the model architecture. This enables precise identification of parameter dynamics under varying token granularity levels, semantic scopes, and generation scales. Building on this analysis, we further uncover sparsity patterns along three critical dimensions-token, layer, and scale-and propose a set of fine-grained optimization strategies tailored to these patterns. Extensive evaluation demonstrates that our approach achieves aggressive acceleration of the generation process while significantly preserving semantic fidelity and fine details, outperforming traditional methods in both efficiency and quality. Experiments on Infinity-2B and Infinity-8B models demonstrate that ToProVAR achieves up to 3.4x acceleration with minimal quality loss, effectively mitigating the issues found in prior work. Our code will be made publicly available.
