Search papers, labs, and topics across Lattice.
MoVerse introduces a novel real-time video world model that generates a fully navigable 3D scene from a single narrow-field-of-view image by first creating a 360掳 panorama and then constructing a persistent 3D Gaussian scaffold. This approach effectively addresses the challenge of limited input data by employing topology-aware diffusion for panorama expansion and Gaussian-conditioned rendering for high-fidelity video output. The system achieves interactive scene roaming at 8 FPS on a single NVIDIA RTX 4090 GPU, showcasing a significant advancement in single-image world modeling and interactive video generation.
Transforming a single image into a fully navigable 3D world in real-time could revolutionize how we interact with visual environments.
We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360$^\circ$ panorama with topology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, a Gaussian-conditioned video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train a bidirectional diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.
