Search papers, labs, and topics across Lattice.
This paper introduces HeadCast, a novel framework that optimizes autoregressive video generation by restructuring the Key-Value cache based on the stable behaviors of attention heads, allowing for efficient inference without the need for model retraining. By classifying attention heads into four archetypes鈥擲ink, Dummy, Spatial, and Global鈥擧eadCast creates head-specific pathways that significantly reduce the inference cost, particularly at higher resolutions. The results demonstrate that HeadCast accelerates inference by up to 1.95x at 1080P while maintaining comparable video quality and minimizing flickering artifacts.
HeadCast accelerates autoregressive video generation by up to 1.95x without retraining, preserving long-range consistency while optimizing attention head usage.
Autoregressive (AR) video diffusion models have become a promising paradigm for long and streaming video synthesis, but the continuously growing Key-Value (KV) cache makes attention the dominant inference cost, especially at high resolution where each frame contributes many tokens. Existing remedies either evict the cache with coarse heuristics that cause inter-frame flickering, or require model re-training. We propose HeadCast, a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's attention heads exhibit stable, heterogeneous behaviors. After a short warm-up, HeadCast performs a one-time classification at the maximum-noise step that sorts every head into one of four archetypes: Sink, Dummy, Spatial, and Global, and restructures the monolithic KV cache into head-specific pathways. Crucially, it retains the Global heads that preserve the long-range temporal consistency aggressive eviction destroys. Because the Spatial pathway operates on a fixed-size grid, its savings grow with resolution: across state-of-the-art AR models, HeadCast accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and largely flicker-free. Code is available at https://github.com/sjlgaga/HeadCast .
