Search papers, labs, and topics across Lattice.
This paper introduces LayerRecall, a state-conditioned memory router designed to enhance long-horizon consistency in video generation by selectively retrieving historical context relevant to specific layers of the model. The method addresses the limitations of existing memory mechanisms that fail to effectively utilize nonlocal history, demonstrating that distinct video DiT layers have varying preferences for context. Experimental results show that LayerRecall outperforms existing methods on multiple benchmarks while maintaining local continuity, highlighting its ability to facilitate memory-guided self-correction in video generation tasks.
LayerRecall boosts long-range video generation consistency by intelligently routing historical context to specific model layers, outperforming traditional methods without sacrificing local detail.
Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.
