Search papers, labs, and topics across Lattice.
This paper introduces EGM-Det, an entropy-guided multimodal adaptive fusion framework designed to enhance UAV object detection using both RGB and infrared imagery. By employing a dual-stream architecture and an innovative Entropy Offset Gate Fusion module, EGM-Det dynamically adjusts the fusion of features based on spatially varying modality reliability, leading to more effective integration of RGB and IR data. The method achieves state-of-the-art performance on multiple benchmarks, significantly outperforming existing approaches, particularly with a more than 10 percentage point improvement on the VEDAI dataset.
EGM-Det achieves over 10% better accuracy in UAV object detection by intelligently fusing RGB and infrared data based on real-time reliability assessments.
Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.
