Search papers, labs, and topics across Lattice.
This paper introduces a novel approach that integrates a physics-grounded neural world model with a Large Language Model (LLM) to enhance navigation planning for Autonomous Underwater Vehicles (AUVs) and Autonomous Surface Vehicles (ASVs) near offshore wind farms. By employing a three-phase gradient-based trajectory optimizer and a Model Predictive Controller, the method ensures safe navigation with zero predicted collisions across various marine environments. The results demonstrate a significant reduction in goal-distance error by 70-93% compared to traditional ungrounded models, showcasing the effectiveness of the world model in real-world scenarios.
Achieving collision-free navigation for AUVs and ASVs near complex offshore wind infrastructures, this approach cuts goal-distance error by up to 93% through physics-informed planning.
Large language models can turn a natural-language mission into a sequence of robot actions, but they do not have a sense of physics: they cannot judge how long a command should run, or whether it will make the robot drift into an obstacle. We proposed the use of a world model to expand the capabilities of Large Language model-based planners. Our method has three components: a physics-grounded neural world model, a three-phase gradient-based trajectory optimizer, and a Model Predictive Controller (MPC)-style closed-loop replanner with a trust-region guard. The language model decides what to do, and the world model decides how long, whether that means driving eight thrusters through 6 DOF or two differential thrusters through 3 DOF. We evaluate two marine vehicle classes operating near offshore wind infrastructure: a 6-DOF Autonomous Underwater Vehicle (AUV) and a 3-DOF differential-drive Autonomous Surface Vehicle (ASV). In five benchmark missions per platform, both vehicles reach every goal with zero predicted collisions, and both transfer to GazeboSim under ocean current, waves, and thruster dynamics, remaining collision-free and cutting GazeboSim goal-distance error versus the ungrounded baseline by 70-82% (ASV) and roughly 93% (AUV), after a residual fine-tuning pass that separately reduces surrogate rollout Root Mean Square Error (RMSE) by 60% (AUV) and 69% (ASV). For the ASV we further demonstrate a Vision language model (VLM)-assisted semantic-mapping pipeline that extracts obstacles and environmental context from satellite imagery, nautical charts, and forecast Application Programming Interface (API) instead of onboard sensors, reaching 96% navigability accuracy as a drop-in replacement for hand-specified obstacle geometry.
