Search papers, labs, and topics across Lattice.
This paper introduces the Laplacian Decoupled Feature Enhancement block (LDFE) to improve dual-stream CNN-based RGB-IR object detection by effectively fusing features from different stages of the backbone. The LDFE employs a novel approach that separates features into global and local components, utilizing a combination of global state space enhancement and local convolutional correlation techniques to enhance feature representation while suppressing noise. The proposed method achieves significant performance gains, surpassing state-of-the-art benchmarks by up to 6.2% mAP across multiple datasets, demonstrating its effectiveness in extreme conditions.
LDFE achieves up to 6.2% mAP improvement over state-of-the-art methods by innovatively decoupling and enhancing features from RGB and IR modalities.
The complementary information between RGB and IR images can significantly enhance object detection performance under extreme conditions. Existing methods prefer dual-stream CNN backbones built upon YOLO for feature extraction and focus on the design of feature fusion. In this paper, we introduce the Laplacian Decoupled Feature Enhancement block (LDFE) to fuse features from different stages of the dual-stream CNN backbone. By design, LDFE simultaneously considers the characteristics of modalities and structures for feature fusion by employing global-local decomposition, denoising, fusion, and reconstruction, sequentially. The LDFE first separates features into global and local components based on Laplacian Pyramid, and then performs denoising and fusion based on Global State Space Enhancement module (GS2E) and Local Convolutional Correlation Enhancement module (LC2E) separately. Specifically, the GS2E conducts a two-branch architecture for the main and auxiliary modalities. It dynamically suppresses noise in the main modality through cross-modal attention derived from the auxiliary modality, while employing a State Space Model to capture long-range dependencies within the global feature representations of the main modality. To obtain bidirectional interaction, the two modalities systematically alternate their main/auxiliary roles. Moreover, the LC2E suppresses noise in local features and leverages spatial and channel dimension along with triple convolution to extract fine-grained details for fusion. These innovative designs achieve a significant performance improvement, with mAP surpassing the SOTA methods 6.2%, 3.7%, 4.7%, 2.3%, 4.1% and 2.0% on M3FD, DroneVehicle, LLVIP, FLIR-Aligned, KAIST and VEDAI datasets,respectively.
