Search papers, labs, and topics across Lattice.
This paper introduces the Energy-Structured Latent World Model (ELWM), which enhances physically consistent motion planning in embodied AI by explicitly incorporating energy and momentum into the latent state representation. By training on multimodal RGB-D and inertial data, the model ensures causal transitions and improves navigation policies through Physics-Conditioned Neural Time Fields (PC-NTF). The results show significant performance gains, including a reduction in motion-prediction error and an increase in navigation success rates, while also minimizing physical collisions, demonstrating the effectiveness of integrating physical structures into latent models for safer motion planning.
Embedding explicit physical structures into latent spaces can bridge the gap between predictive models and safe, dynamically feasible motion planning.
Physically consistent motion planning remains a fundamental challenge in embodied AI, as generated trajectories must strictly conform to real-world execution dynamics. While latent world models offer a promising approach by predicting these dynamics, existing methods learn unconstrained future representations where absorbed physics remains implicit. Therefore, they fail to form reusable physical knowledge, which compromises reliability in unpredictable open-world navigation. To address this, we propose a novel Energy-Structured Latent World Model (ELWM). Our key idea is to structure the ELWM latent state to explicitly carry energy and momentum, ensuring strictly causal transitions via dissipation and control ports. Trained on multimodal RGB-D and inertial interaction histories, our model guarantees physically consistent predictions. We further implement this for motion planning by constructing Physics-Conditioned Neural Time Fields (PC-NTF), a key technical cornerstone that integrates ELWM into an arrival time field via the Eikonal equation to yield a physically-informed navigation policy. Across held-out scenes, our evaluation reveals significant improvements. Compared to generic latent models, PC-NTF reduces 0.8-s motion-prediction NRMSE from 0.36 to 0.29. Against Active Neural Time Fields, it improves navigation success from 81.3% to 89.7% and SPL from 0.64 to 0.73, while cutting the physical collision rate from 12.1% to 5.8% and the Eikonal residual from 0.083 to 0.031. Beyond these targeted gains, our results demonstrate that embedding explicit physical structures into latent spaces intrinsically bridges the gap between predictive world models and safe, dynamically feasible motion planning.
