Search papers, labs, and topics across Lattice.
This paper introduces a sparse fusion mechanism for RGB-T object detection that significantly reduces computational costs while maintaining detection accuracy. By first identifying high-recall regions of interest (RoIs) using lightweight single-modality models, the method then applies a focused feature fusion approach to refine these proposals. Extensive experiments demonstrate that this two-stage framework achieves competitive performance with fewer parameters and lower computational demands, making it scalable to high-resolution images.
Sparse fusion in RGB-T detection slashes computational costs while preserving accuracy, enabling efficient processing of high-resolution images.
RGB-T detectors leverage the complementary strengths of visible and thermal infrared modalities, achieving robust performance under challenging conditions. Many of them resort to heavy dual backbones and exhaustive cross-modality fusion across the entire image, leading to impractically high computational costs. We observe that most image regions are smooth backgrounds (e.g., sky, ground) that can be easily handled by lightweight single-modality models. In light of this observation, we propose a sparse fusion mechanism for efficient RGB-T detection: first rapidly scanning the image to identify the proposals and then carefully examining the remaining sparse proposals via feature fusion. We propose a two-stage framework to instantiate this mechanism, which performs detection in two stages: 1) a lightweight and modality-specific detection stage that produces high-recall RoIs, and 2) a fusion-driven examination and refinement stage that filters out the false positives and refines the bounding boxes. This design enables the detector to adaptively allocate more computational resources to the potential foregrounds, improving the efficiency while ensuring detection accuracy. Extensive experiments show that our method achieves competitive performance with substantially fewer parameters and lower cost, while maintaining strong scalability to high-resolution images.
