Search papers, labs, and topics across Lattice.
This paper introduces RECAP-Forcing, a novel approach to long autoregressive video generation that organizes memory based on appearance novelty rather than recency. By retaining the key-value (KV) cache associated with newly introduced content, the method ensures long-range consistency in video generation, addressing the limitations of traditional memory management that prioritizes recent frames. The results show that RECAP-Forcing significantly enhances visual quality and semantic fidelity compared to existing memory techniques, all without requiring additional learnable parameters.
Memory management in video generation just got a makeover鈥攑rioritizing novelty over recency leads to better visual coherence and fidelity.
Long autoregressive video generation faces a fundamental memory challenge: with a finite attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing memory by appearance novelty. A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize memory by retaining the KV cache associated with newly appearing content--such as entering subjects, disoccluded regions, and newly introduced scenes--at the moment it first becomes visible, prioritizing novelty over recency. Memory should scale with the amount of newly introduced content, rather than with video length. This appearance-indexed memory makes long-range consistency an explicit property of the memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a video, when all visible content is novel, an attention sink preserves the initial scene. As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing memory methods.
