Search papers, labs, and topics across Lattice.
The paper introduces WorldKV, a training-free framework for maintaining persistent world consistency in autoregressive video diffusion models. It addresses the trade-off between memory fidelity and throughput by employing World Retrieval, which selectively retrieves relevant KV-cache chunks based on camera/action correspondence, and World Compression, which prunes redundant tokens within chunks. Experiments on Matrix-Game-2.0 and LingBot-World-Fast demonstrate that WorldKV achieves comparable or superior memory fidelity to full-KV attention at twice the throughput, rivaling memory-trained baselines without fine-tuning.
Get persistent world memory in your video diffusion model for free: WorldKV matches full KV-cache fidelity at 2x throughput without any training.
Autoregressive video diffusion models have enabled real-time, action-conditioned world generation. However, sustaining a persistent world, where revisiting a previously seen viewpoint yields consistent content, remains an open problem. Full KV-cache attention preserves this consistency but breaks real-time constraints: memory footprint and attention cost grow linearly with rollout length. Sliding window inference restores throughput but discards long-term consistency. We propose WorldKV, a training-free framework with two components: World Retrieval and World Compression. World Retrieval stores evicted KV-cache chunks in GPU/CPU memory and selectively retrieves scene-relevant chunks via camera/ action correspondence, inserting them back into the native attention window without re-encoding. World Compression prunes redundant tokens within each chunk via key-key similarity to an anchor frame, halving per-chunk storage to fit 2x more history under a fixed budget. On Matrix-Game-2.0 and LingBot- World-Fast, WorldKV matches or exceeds full-KV memory fidelity at roughly 2x the throughput, and is competitive with memory-trained baselines without any fine-tuning. Project Page: https://cvlab-kaist.github.io/WorldKV/
