Search papers, labs, and topics across Lattice.
This paper introduces LongLive-RAG, a novel retrieval-augmented framework for autoregressive video generation that mitigates error accumulation and identity drift in long-horizon synthesis. By treating previously generated latents as a dynamic, searchable history, the method allows for conditioning on non-local context rather than being constrained to a sliding-window approach. Experimental results demonstrate significant improvements in video quality and the best average rank on the VBench-Long benchmark across various autoregressive backbones and generation lengths.
LongLive-RAG transforms long video generation by enabling the use of a searchable memory of past latents, drastically reducing error accumulation.
Autoregressive (AR) video diffusion enables variable-length synthesis, but long-horizon generation often suffers from accumulated errors and identity drift. For efficiency, existing methods commonly adopt sliding-window attention during generation. This creates an irreversible generation trajectory: once the active window accumulates appearance errors, subsequent generations can only condition on this degraded trajectory and drift further away. We address this limitation by formulating long video generation as a retrieval-augmented generation (RAG) problem. Rather than relying solely on the recent window, we treat previously generated latents as a dynamic, searchable history. We propose LongLive-RAG, a general retrieval framework for AR video generation. At each new block, LongLive-RAG uses a query embedding to retrieve relevant historical latents. This lightweight retrieval step adds only a small overhead relative to generation and lets the generator condition on non-local context instead of only the recent window. To make retrieval more discriminative, we introduce the Window Temporal Delta Loss that suppresses redundant local similarity and encourages embeddings to capture meaningful temporal changes. Together, these components help reduce error accumulation caused by sliding-window attention. Experiments across multiple AR backbones and generation lengths show improved long-video quality and the best average VBench-Long rank. To our knowledge, among open-ended AR long video generation methods, LongLive-RAG is the first to formulate self-generated latent history as content-addressable retrieval memory. Code is available at https://github.com/qixinhu11/LongLive-RAG.
