Search papers, labs, and topics across Lattice.
Veda introduces a distilled sparse attention mechanism for video diffusion transformers that addresses the quadratic cost of self-attention without sacrificing generation quality. By formulating tile selection as a reconstruction problem from full attention and using statistics-aware tile scoring with head-aware tiling, Veda achieves aggressive sparsity while maintaining alignment with the tile-wise geometry of full attention. Experiments on Waver and Wan2.1 demonstrate significant speedups (up to 5.1x end-to-end and 10.5x self-attention) when generating high-resolution, long videos, with gains increasing with sequence length.
Surprisingly, high sparsity in video diffusion models doesn't degrade generation quality if the sparse mask accurately mimics the tile-wise geometry of full attention.
Scaling Diffusion Transformers to generate high-resolution, long videos is constrained by the quadratic cost of self-attention, and existing sparse attention methods degrade under high sparsity. We show empirically that generation quality is determined not by the sparsity ratio itself, but by how well the sparse mask aligns with the tile-wise geometry of full attention. Based on this insight, we propose Veda, a distilled sparse attention framework that formulates tile selection as an explicit reconstruction problem from full attention. Veda integrates statistics-aware tile scoring with head-aware tiling to reduce estimation error and structural mismatch, enabling aggressive sparsity. A hardware-efficient tile-skipping kernel converts theoretical sparsity into practical wall-clock speedups. Experiments on large video diffusion models, including Waver and Wan2.1, demonstrate substantial acceleration with no noticeable degradation in generation quality. To generate 720P 10-second videos on Waver-T2V-12B, Veda achieves a 5.1$\times$ end-to-end speedup and a 10.5$\times$ self-attention speedup, reducing attention overhead from 92% to 50%. Notably, the gains increase with sequence length, indicating that Veda scales favorably with spatiotemporal resolution across models.
