Search papers, labs, and topics across Lattice.
The paper introduces UAVReason, a large-scale multimodal benchmark for nadir-view UAV scenarios, addressing the performance gap of VLMs when applied to high-altitude UAV imagery due to domain shift. UAVReason consolidates over 273K VQA pairs, temporal sequences, and cross-modal generation samples to probe 22 diverse reasoning types and evaluate high-fidelity generation across RGB, depth, and segmentation modalities. Through multi-task learning, the authors establish a strong baseline model, demonstrating improved UAV-native performance compared to general-domain VLMs.
General-purpose vision-language models stumble when looking down from drones, but a new benchmark reveals that multi-task learning can substantially improve their aerial reasoning skills.
Vision-Language models (VLMs) have demonstrated remarkable capability in ground-view visual understanding but often fracture when deployed on high-altitude Unmanned Aerial Vehicles (UAVs). The failure largely stems from a pronounced domain shift, characterized by tiny and densely packed objects, repetitive textures, and ambiguous top-down orientations. These factors severely disrupt semantic grounding and hinder both spatial reasoning and controllable generation. To bridge this critical gap, we introduce UAVReason, the first unified large-scale multi-modal benchmark dedicated to nadir-view UAV scenarios, derived from a high-fidelity UAV simulation platform. In contrast to existing UAV benchmarks, which are largely siloed and focus on single tasks like object detection or segmentation, UAVReason uniquely consolidates over 273K Visual Question Answering (VQA) pairs, including 23.6K single frames with detailed captions, 68.2K 2-frame temporal sequences, and 188.8K cross-modal generation samples. The benchmark probes 22 diverse reasoning types across spatial and temporal axes while simultaneously evaluating high-fidelity generation across RGB, depth, and segmentation modalities. We further establish a strong, unified baseline model via multi-task learning. Extensive experiments validate the efficacy of our unified approach across diverse metrics, such as EM/F1 for VQA, mIoU for segmentation, and CLIP Score for generation. These results indicate limitations of general-domain vision-language models and show that unified multi-task learning substantially improves UAV-native performance. All data, code, and evaluation tools will be publicly released to advance UAV multimodal research.
