Search papers, labs, and topics across Lattice.
This paper introduces ProtoHGF-Net, a novel framework for RGB-Thermal object detection that shifts from dense cross-modal interactions to prototype-level semantic interactions. By employing Prototype HyperGraph Fusion and Teacher-Mask Calibration Distillation, the method effectively reduces background interference and enhances the learning of target-relevant representations. Experimental results on multiple datasets show that ProtoHGF-Net achieves state-of-the-art performance, with mean Average Precision (mAP) scores of 85.9%, 88.2%, and 79.1% across different benchmarks.
Redefining cross-modal fusion at the prototype level, ProtoHGF-Net significantly enhances target detection by suppressing background noise.
RGB-Thermal (RGBT) object detection enables robust perception in complex scenes by leveraging the complementary strengths of visible textures and thermal cues. However, existing methods mainly rely on dense cross-modal interactions over full-resolution features, which inevitably introduce background interference and hinder the learning of target-relevant representations. In this paper, we propose the Prototype HyperGraph Fusion Network (ProtoHGF-Net), a novel framework that redefines cross-modal fusion as prototype-level semantic interaction rather than the dense cross-modal interaction paradigm. Specifically, we design Prototype HyperGraph Fusion to perform cross-modal interaction in a compact prototype-level semantic space. This design enables more selective fusion among target-relevant prototypes. To support this prototype-level fusion, we propose Teacher-Mask Calibration Distillation, which calibrates modality features before fusion using modality-specific teachers and target-aware masks. This strategy suppresses backgrou- nd-dominant responses and produces more target-focused features. Extensive experiments on DroneVehicle, DVTOD, and FLIR demonstrate that ProtoHGF-Net achieves state-of-the-art performance with 85.9\% $mAP_{50}$, 88.2\% $mAP_{50}$, and 79.1\% $mAP_{50}$, respectively. Our code is available at \href{https://github.com/ZiMo-Chen/ProtoHGF}{GitHub}.
