Search papers, labs, and topics across Lattice.
The paper introduces Future Forcing, a novel training-free KV cache policy for autoregressive video generation that addresses the limitations of existing methods in long-horizon consistency. It leverages the observed stationarity of pre-RoPE query distributions in trained AR video models to estimate future query distributions from historical statistics, enabling future-aware cache decisions. By scoring KV cache tokens based on their importance under this future query proxy and merging redundant tokens, Future Forcing achieves significant improvements in subject consistency for long video generation tasks.
Forget short-sighted compression: Future Forcing anticipates future query needs in autoregressive video generation, boosting long-horizon consistency by up to 1.49 on VBench-Long without any training.
Autoregressive (AR) video generation has emerged as a promising paradigm for long-horizon video synthesis, where each frame is generated conditioned on previously generated tokens. To accelerate inference, the KV cache is used to avoid redundant recomputation across generation steps. Nevertheless, its growth with generation length introduces increasing memory and error accumulation, limiting the scalability of AR models to even longer sequences. Existing KV cache compression methods mitigate this issue by selectively retaining only video tokens deemed important. However, most existing methods assess token importance using short-horizon signals derived from the current or historical generation context, making these methods prone to overlooking tokens that appear unimportant at early steps but later become critical for future frames. In this work, we identify an important property of trained AR video models: although RoPE-modulated queries evolve across autoregressive steps, the underlying canonical pre-RoPE query distribution remains remarkably stable throughout the video generation process. This approximate stationarity implies that future query distributions are estimable from historical statistics, enabling principled future-aware cache decisions without any additional training. Building on this insight, we propose Future Forcing, a training-free future-aware KV cache policy for AR video generation. Specifically, Future Forcing first constructs a future query proxy from historical statistics, then scores KV cache tokens by their importance under this proxy, and finally merges redundant token pairs within the affine subspace induced by the future query. Extensive experiments show that Future Forcing improves long-horizon consistency under limited KV caches, achieving up to 1.49 improvement in subject consistency on VBench-Long for 60s generation over existing AR video KV cache policies.
NUS