Search papers, labs, and topics across Lattice.
This paper investigates the limitations of visual memory in interactive video world models, particularly how temporal Rotary Positional Embeddings (RoPE) hinder the retrieval of stored content beyond the training horizon. To overcome these challenges, the authors introduce WorldTrace, a training-free memory framework that maintains addressability in compressed memory by assigning distinct virtual positions to summary slots. The results demonstrate significant improvements in temporal consistency and episodic recall, with WorldTrace-Field enhancing performance by 15.5% and WorldTrace-Landmark by 19.5% on the LoopBench benchmark.
WorldTrace redefines memory management in video world models, achieving up to 19.5% better episodic recall without any retraining.
We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward previously generated frames. However, we find that models can no longer reliably address stored content once rollouts extend beyond the training horizon, because temporal Rotary Positional Embeddings (RoPE) offsets then fall outside the range seen during training and the model struggles to retrieve the relevant visual information through attention. Moreover, naively compressing the cache in the RoPE-rotated space corrupts memory by averaging together incompatible positional phases. To address this, we propose WorldTrace, a training-free memory framework for long-horizon visual persistence. WorldTrace keeps compressed memory addressable by assigning each summary slot a distinct, in-distribution virtual position. Within this addressable cache, we study two memory compression approaches: WorldTrace-Field compresses history for temporal coherence, while WorldTrace-Landmark stores verbatim scene traces at detected transitions for episodic recall. We further introduce LoopBench, a benchmark evaluating whether a compressed cache can reconstruct a previously visited scene after a long detour. WorldTrace-Field improves temporal consistency by +15.5%, and WorldTrace-Landmark improves episodic recall by +19.5% on LoopBench, extending visually persistent generation without retraining.
NVIDIA