Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a hybrid framework that integrates Computer Vision (CV) models with Large Vision-Language Models (LVLMs) to enhance post-disaster building damage assessment using UAV imagery. By decoupling the detection of buildings from the damage assessment process, the framework achieves superior accuracy in classifying building conditions while requiring minimal annotated data for the detection stage. Evaluations on real-world benchmarks, RescueNet and FloodNet, show that this approach outperforms traditional methods by up to 2.1 R² points, providing valuable insights into failure scenarios and future research directions.
A hybrid framework combining CV precision with LVLM reasoning boosts damage assessment accuracy by over 2 R² points while minimizing data requirements.
Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer Vision (CV) models often demand large annotated datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large Vision-Language Models (LVLMs) offer a promising alternative through their strong reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git