Search papers, labs, and topics across Lattice.
This paper introduces RECAP-Forcing, a novel approach to long autoregressive video generation that organizes memory based on the novelty of content appearance rather than temporal recency. By prioritizing the retention of key visual elements as they first appear, the method ensures long-range consistency in video narratives, addressing the limitations of traditional memory management techniques. The results demonstrate significant improvements in visual quality and semantic fidelity compared to existing memory methods, without requiring additional learnable parameters during inference.
Prioritizing content novelty over recency, RECAP-Forcing revolutionizes memory management in long video generation, leading to enhanced visual coherence.
Long autoregressive video generation faces a fundamental memory challenge: with a finite attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing memory by appearance novelty. A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize memory by retaining the KV cache associated with newly appearing content--such as entering subjects, disoccluded regions, and newly introduced scenes--at the moment it first becomes visible, prioritizing novelty over recency. Memory should scale with the amount of newly introduced content, rather than with video length. This appearance-indexed memory makes long-range consistency an explicit property of the memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a video, when all visible content is novel, an attention sink preserves the initial scene. As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing memory methods.
