Search papers, labs, and topics across Lattice.
This paper introduces ImagineUAV, an innovative framework for aerial vision-language navigation that employs cascaded world-action modeling to enhance 6-DoF flight under partial observability. By utilizing a latent video diffusion model to generate future observations conditioned on instructions, the system effectively imagines environmental changes, allowing for more accurate motion inference through an action extractor. The implementation of a kinodynamic planner refines these motions into collision-free trajectories, achieving superior performance to existing VLN and VLA models with only 1.3B parameters, demonstrating the feasibility of imagination-driven navigation in real-world scenarios.
ImagineUAV outperforms existing models in aerial navigation by leveraging an innovative imagination-driven approach that predicts future environments.
Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability. While Vision-Language-Action (VLA) models excel at semantic reasoning, they suffer from brittleness due to geometric inconsistency and dynamics mismatch. To address this, we propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling. Instead of direct regression, ImagineUAV employs a latent video diffusion model to generate instruction-conditioned future observations, explicitly imagining environmental evolution, from which 6-DoF motions are inferred via an action extractor. A kinodynamic planner then refines these estimates into collision-free trajectories. Additionally, a step-distilled inference pipeline ensures real-time execution. With only 1.3B parameters, ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights, validating the practicality of imagination-driven aerial navigation.
