Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To overcome the repetitive, template-driven questions and static-image constraints of prior aerial benchmarks, the authors establish ReVA, a remote sensing video question answering dataset comprising 2,438 drone videos (580K frames) and 22K question-answer pairs spanning 11 spatiotemporal tasks. The benchmark models real-world UAV dynamics across 18 global cities, curating diverse reasoning challenges via a semi-automated MLLM/LLM generation pipeline backed by human verification. Extensive evaluations across 23 proprietary and open-source Video-LLMs reveal that current state-of-the-art models suffer from fundamental deficiencies in aerial spatiotemporal reasoning and dynamic scene understanding.
Even state-of-the-art Video-LLMs fail to generalize to dynamic aerial environments once evaluation moves beyond repetitive question templates and static satellite imagery.
Multimodal Large Language Models (MLLMs) have demonstrated remarkable advances in remote sensing. However, existing remote sensing multimodal reasoning benchmarks exhibit two critical limitations: they rely on (i) template-driven questions, which causes repetitive questions; and (ii) static images that fail to capture the inherent temporal nature of drone/UAV videos. This leaves systematic evaluation of remote sensing video reasoning largely unexplored. To address this gap, we introduce ReVA, a new dataset for remote sensing video question answering, designed to assess spatiotemporal, scene-centric, and reasoning-oriented capabilities of MLLMs. ReVA comprises 2,438 drone videos spanning 18 cities worldwide (580K frames) and 22K high-quality question-answer pairs across 11 challenging QA tasks. We develop a semi-automatic annotation pipeline that leverages Text LLMs and MLLMs for question-answer generation with human verification. We evaluate 23 proprietary and open-source Video LLMs on ReVA, exposing fundamental limitations of current models. These findings position ReVA as a critical benchmark toward better remote sensing video understanding and temporal reasoning capabilities for real-world deployments. Our code and dataset are available at: https://github.com/zyaocoder/ReVA