Search papers, labs, and topics across Lattice.
This paper introduces WorldFly, a world-model-based Vision-Language-Action (VLA) framework designed to enhance UAV navigation in challenging urban environments characterized by severe occlusions and drastic viewpoint transitions. By employing a dual-branch coupled flow matching mechanism, WorldFly generates future video predictions alongside navigation actions, allowing for improved spatial understanding and decision-making under partial observability. Extensive evaluations on a newly constructed Urban Canyon Traversal Benchmark reveal that WorldFly significantly outperforms existing baselines, particularly in unseen environments, highlighting the importance of spatial imagination in aerial navigation tasks.
WorldFly's innovative approach to integrating world models enables UAVs to navigate complex urban landscapes with unprecedented robustness.
End-to-end Vision-Language-Action (VLA) models have shown promise in UAV navigation. However, existing approaches typically rely on historical observations to directly predict actions, often struggling in dense urban environments where severe occlusions and sharp turns result in drastic viewpoint transitions. We argue that the ability to "imagine" future states -- inherent in World Models -- is critical for robust decision-making under such partial observability. To address this, we construct a challenging Urban Canyon Traversal Benchmark, specifically designed to evaluate spatial understanding in scenarios characterized by severe occlusions and drastic viewpoint transitions. To this end, we propose WorldFly, a novel world-model-based VLA framework that employs a dual-branch coupled flow matching mechanism to jointly generate future video predictions and navigation actions, thereby explicitly guiding the agent's policy via spatial imagination. Extensive evaluations on our benchmark demonstrate that WorldFly outperforms other baselines, particularly in unseen environments, validating the effectiveness of integrating world models into embodied aerial agents.
Tsinghua AI