Search papers, labs, and topics across Lattice.
This paper introduces IOI, a hybrid interactive world model that combines analytical kinematic priors with learned physical dynamics to enhance the fidelity of embodied agents in interactive environments. By decoupling kinematics from physics, IOI provides precise control alignment and visually realistic feedback, overcoming limitations of purely data-driven methods that suffer from spatiotemporal drift. Experimental results demonstrate that IOI achieves state-of-the-art performance on the RoboTwin benchmark, showcasing robust zero-shot generalization and reliable policy evaluation that closely aligns with ground-truth physics simulators.
IOI achieves state-of-the-art simulation performance by decoupling deterministic motion from stochastic physical interactions, enabling robust zero-shot generalization to unseen tasks.
Developing generalist embodied agents requires interactive environments providing visually realistic feedback and accurate action-conditioned dynamics. Interactive world models address this by simulating such complex dynamics. However, purely data-driven methods struggle to ensure precise control alignment and physically plausible visual feedback due to a lack of explicit structural constraints. To address this, we propose IOI, a hybrid interactive world model integrating analytical kinematic priors with learned physical dynamics. Unlike data-driven approaches prone to spatiotemporal drift, IOI introduces explicit kinematic guidance, computing forward kinematics from action sequences for accurate motion trajectories. These trajectories are rendered into synchronized front, side, and top orthographic projections, eliminating the need for extrinsic camera calibration. A Multi-view Kinematic Aggregation and Injection module fuses these geometric cues and injects them into the video generator, providing geometry-consistent guidance. Conditioning video generation on these deterministic trajectories establishes a synergy between the analytical simulator and the world model. Decoupling deterministic motion into the kinematic prior frees the generator to model stochastic physical interactions. Experiments on the RoboTwin benchmark validate IOI across kinematic fidelity, out-of-distribution (OOD) generalization, and policy evaluation. IOI achieves state-of-the-art simulation performance and robust zero-shot generalization to unseen OOD tasks. Furthermore, IOI serves as a reliable policy evaluator, yielding success rates closely aligning with ground-truth physics simulators. On real-world platforms, policies trained on IOI-synthesized data match those trained on teleoperation demonstrations, solidifying its practical value for embodied policy learning.
