Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
MosaiChunk addresses the long-horizon context limitation in autoregressive video generation by compositing spatio-temporal memory from selected historical key-value entries. This mechanism allows a frozen generator to recover fine-grained visual details of objects and scenes that reappear after falling out of the active context window. Experiments on the new RememBench benchmark demonstrate MosaiChunk's superior revisit consistency compared to sliding-window and whole-chunk retrieval methods under equivalent memory budgets.
Recovering lost visual details in long-horizon video generation is now possible by compositing non-contiguous spatio-temporal memory, even with a frozen generator.
Long-horizon autoregressive video generation is limited by a finite context window. When an object or scene falls out of context, its fine-grained visual details may be lost and difficult to recover upon reappearance. To retain access to such visual details, we introduce MosaiChunk, a spatio-temporal memory mechanism that composes a mosaic of selected historical key-value (KV) entries across space and time. Our approach is motivated by the observation that a frozen video generator can directly consume such non-contiguous historical KV and recover the corresponding visual content. We therefore keep the generator fixed and learn only a lightweight router that determines which historical sections to include in the mosaic under a fixed active-memory budget. We further introduce RememBench, a benchmark of long-horizon revisits with prompt-driven text-to-video (T2V) and camera-driven image-to-video (I2V) splits. Our experiments show that MosaiChunk consistently improves revisit consistency over both sliding-window inference and whole-chunk retrieval under matched memory budgets, across both T2V and I2V settings.