Search papers, labs, and topics across Lattice.
INSPATIO-WORLD is introduced, a real-time framework for generating interactive 4D scenes from a single reference video. The core innovation is a Spatiotemporal Autoregressive (STAR) architecture with an Implicit Spatiotemporal Cache for global consistency and an Explicit Spatial Constraint Module for physically plausible interactions. Joint Distribution Matching Distillation (JDMD) is used to improve fidelity by regularizing with real-world data distributions, leading to state-of-the-art performance on the WorldScore-Dynamic benchmark.
Finally, a real-time 4D world simulator exists that allows for consistent and controllable scene evolution from a single monocular video.
Building world models with spatial consistency and real-time interactivity remains a fundamental challenge in computer vision. Current video generation paradigms often struggle with a lack of spatial persistence and insufficient visual realism, making it difficult to support seamless navigation in complex environments. To address these challenges, we propose INSPATIO-WORLD, a novel real-time framework capable of recovering and generating high-fidelity, dynamic interactive scenes from a single reference video. At the core of our approach is a Spatiotemporal Autoregressive (STAR) architecture, which enables consistent and controllable scene evolution through two tightly coupled components: Implicit Spatiotemporal Cache aggregates reference and historical observations into a latent world representation, ensuring global consistency during long-horizon navigation; Explicit Spatial Constraint Module enforces geometric structure and translates user interactions into precise and physically plausible camera trajectories. Furthermore, we introduce Joint Distribution Matching Distillation (JDMD). By using real-world data distributions as a regularizing guide, JDMD effectively overcomes the fidelity degradation typically caused by over-reliance on synthetic data. Extensive experiments demonstrate that INSPATIO-WORLD significantly outperforms existing state-of-the-art (SOTA) models in spatial consistency and interaction precision, ranking first among real-time interactive methods on the WorldScore-Dynamic benchmark, and establishing a practical pipeline for navigating 4D environments reconstructed from monocular videos.
