Search papers, labs, and topics across Lattice.
The paper introduces FlameVQA, a novel benchmark for visual question answering (VQA) in UAV-based wildfire monitoring, which integrates RGB imagery with radiometric thermal data to enhance reasoning capabilities in complex aerial environments. By employing a combination of MLLM-assisted annotation and deterministic thermal rules, the authors ensure high label reliability across a diverse set of tasks, including detection and flight planning. Evaluation of existing MLLMs on this benchmark reveals strong performance with cross-modal cues but highlights significant limitations in challenging conditions, indicating a need for domain-specific model adaptations.
Current MLLMs struggle with wildfire monitoring, showing notable failures in critical tasks like presence detection under smoke and coverage estimation.
Wildfire monitoring from UAVs requires reliable reasoning over complex aerial scenes, where smoke, scale variation, and occlusions often limit RGB-only interpretation. We introduce FlameVQA, a multiple-choice visual question answering benchmark for UAV-based wildfire intelligence built on FLAME 3, leveraging paired RGB imagery and radiometric thermal TIFFs for temperature-grounded, safety-critical reasoning. FlameVQA includes 34 multiple-choice questions per image spanning six operational capability groups, covering tasks such as detection, localization, distribution/coverage estimation, cross-modal reasoning, and flight planning. To ensure label reliability, we combine MLLM-assisted annotation with deterministic thermal rules and cross-question consistency checks, followed by human auditing. We also evaluate representative MLLMs on FlameVQA to provide baselines for future work. Results show strong performance when explicit cross-modal cues are available, but notable failures on presence detection under heavy smoke and on coverage estimation. These findings suggest that current MLLMs require domain-specific adaptation to better support disaster and wildfire monitoring. The dataset and benchmark code are open-source at github.com/mobiiin/WildFire_VQA
