Search papers, labs, and topics across Lattice.
This paper introduces SPADE, a training-free sparse attention engine designed to enhance the efficiency of video diffusion transformers (vDiTs) by implementing input-adaptive sparsity. By utilizing a three-part architecture that includes a specification for dynamic masks, a runtime scheme generation, and an optimized executor, SPADE significantly improves the speed and sparsity of attention mechanisms. The results demonstrate that SPADE accelerates attention processing by 2.26x-3.40x and overall inference by 1.49x-1.80x without compromising output quality across various video generation tasks.
SPADE achieves up to 3.40x faster attention processing in video diffusion models while maintaining high-quality outputs, revolutionizing the efficiency of video generation.
Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost, making inference prohibitive at video-token scales. The challenge is input-adaptive sparsity: selecting critical Q/K/V tokens with negligible overhead and executing them for end-to-end gains. We present SPADE, a training-free sparse-attention engine of three parts: (i) vDiT-SSR, a specification defining 3D blocking candidates and formalizing dynamic masks via Summarizer/Estimator expressions; (ii) runtime scheme generation using SICS and a head-wise policy; and (iii) an executor with low-overhead index search, flash block-sparse attention, and kernel grouping. Across Hunyuan-Video and Wan 2.1/2.2 for text-to-video and image-to-video generation, SPADE raises sparsity and speed while preserving quality, accelerating attention by 2.26x-3.40x and end-to-end inference by 1.49x-1.80x. Our code is open-sourced at https://github.com/6somehow/DAC-SPADE.
