Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
WildFireVQA, a new large-scale VQA benchmark, is introduced to evaluate multimodal reasoning for aerial wildfire monitoring, integrating RGB imagery with radiometric thermal data. The benchmark contains 207,298 multiple-choice questions across various wildfire-related tasks, generated using a combination of MLLM-based answer generation, sensor-driven deterministic labeling, and manual verification. Experiments with representative MLLMs reveal that RGB data remains the most informative modality, but retrieval-augmented thermal context improves performance for stronger models, highlighting the potential and limitations of current MLLMs in this domain.
Current multimodal LLMs struggle to effectively leverage radiometric thermal data for safety-critical wildfire monitoring, despite its potential to improve situational awareness.
Wildfire monitoring requires timely, actionable situational awareness from airborne platforms, yet existing aerial visual question answering (VQA) benchmarks do not evaluate wildfire-specific multimodal reasoning grounded in thermal measurements. We introduce WildFireVQA, a large-scale VQA benchmark for aerial wildfire monitoring that integrates RGB imagery with radiometric thermal data. WildFireVQA contains 6,097 RGB-thermal samples, where each sample includes an RGB image, a color-mapped thermal visualization, and a radiometric thermal TIFF, and is paired with 34 questions, yielding a total of 207,298 multiple-choice questions spanning presence and detection, classification, distribution and segmentation, localization and direction, cross-modal reasoning, and flight planning for operational wildfire intelligence. To improve annotation reliability, we combine multimodal large language model (MLLM)-based answer generation with sensor-driven deterministic labeling, manual verification, and intra-frame and inter-frame consistency checks. We further establish a comprehensive evaluation protocol for representative MLLMs under RGB, Thermal, and retrieval-augmented settings using radiometric thermal statistics. Experiments show that across task categories, RGB remains the strongest modality for current models, while retrieved thermal context yields gains for stronger MLLMs, highlighting both the value of temperature-grounded reasoning and the limitations of existing MLLMs in safety-critical wildfire scenarios. The dataset and benchmark code are open-source at https://github.com/mobiiin/WildFire_VQA.