Search papers, labs, and topics across Lattice.
SPW-Nav is a streaming panoramic world model that generates 2K 360-degree video from a single panorama based on language instructions, enabling real-time interactive 3D scene exploration. It achieves this by decoupling spherical rotation, pose-aligning translation conditioning, and employing a multi-term memory with a few-step generator to maintain scene coherence across instruction changes. SPW-Nav outperforms prior methods in camera-following accuracy and video quality, and supports dynamic instruction switching.
Language instructions now directly control real-time, high-fidelity 360掳 video generation, enabling fluid interactive exploration from a single panorama.
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
