Search papers, labs, and topics across Lattice.
This paper introduces Token Radius Attention (TRA), a novel framework that optimizes attention computation in Video Diffusion Transformers (VDiTs) by mapping query entropy to a dynamic token budget, allowing for efficient video generation. By leveraging the correlation between retained attention density and attention entropy, TRA reduces the number of attention interactions to just 9-19%, achieving significant speedups of 1.56x to 2.05x while maintaining competitive generation quality. The approach circumvents the need for explicit key ranking, streamlining the attention process and enhancing computational efficiency in video generation tasks.
TRA achieves up to 2.05x faster video generation by intelligently mapping query entropy to token-specific attention budgets, revolutionizing efficiency in VDiTs.
Video Diffusion Transformers (VDiTs) enable high-fidelity generation but incur quadratic cost from dense 3D self-attention. Existing head- and block-level sparse methods share computation budgets across queries, overlooking token-specific attention demand. We observe that retained density varies across queries yet correlates log-linearly with attention entropy, while dominant interactions form query-centered neighborhoods with token-dependent radii. Based on these findings, we propose Token Radius Attention (TRA), a training-free framework that maps query entropy to an analytic token budget and converts it into a temporally decayed radius without explicit key ranking. Fused entropy extraction, warm-up reuse, and block-sparse mask construction further reduce overhead. Across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations, TRA retains only 9-19% of attention interactions and achieves 1.56x-2.05x speedup with competitive generation quality. Code is available at https://github.com/IF-LAB-PKU/Token-Radius-Attention.
