Search papers, labs, and topics across Lattice.
This paper introduces Compression and Retrieval (CaR), an innovative attention-driven implicit memory retrieval mechanism designed to enhance long-term memory consistency in video world models across complex camera trajectories. By integrating viewpoint information through positional encoding and employing a lightweight context compression network, CaR efficiently retrieves relevant memory with minimal computational overhead. Experimental results on the newly constructed SceneFly dataset reveal that CaR achieves state-of-the-art performance and demonstrates robust generalization capabilities in open-domain scenarios.
Achieving state-of-the-art performance in long-horizon video world models, CaR flexibly retrieves memory while minimizing computational costs.
Video world models hold promise for simulating interactive environments, yet maintaining consistent long-term memory across complex camera trajectories remains a critical challenge. Existing methods typically rely on computationally expensive context scaling or rigid heuristic retrieval mechanisms, which lacks generalization to varying camera trajectories and environments. In this paper, we propose Compression and Retrieval (CaR), an attention-driven implicit memory retrieval mechanism to overcome these limitations. By injecting viewpoint information via positional encoding, our method performs flexible memory retrieval through attention computation. To efficiently process extended contexts with minimal computational overhead, we further introduce a lightweight context compression network. Furthermore, we construct SceneFly, a large-scale synthetic dataset featuring realistic camera trajectories and frame-level annotations to train and evaluate long-horizon video world models. Extensive experiments demonstrate that our approach achieves state-of-the-art results on established benchmarks and exhibits strong generalization to open-domain scenes.
