Search papers, labs, and topics across Lattice.
This paper introduces a mask-guided multi-modality image fusion (MMIF) method specifically designed to enhance image representation under adverse weather conditions by integrating feature restoration and interaction. By leveraging a novel "Pseudo Ground Truth" for training and a mask generation mechanism that quantifies the contributions of different modalities, the approach effectively improves feature learning and reduces overfitting. Extensive experiments show that this method outperforms existing state-of-the-art techniques in both visual quality and quantitative metrics across various datasets.
Adverse weather conditions can significantly degrade image quality, but our mask-guided MMIF method achieves superior feature restoration and interaction, setting a new benchmark in the field.
Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask-guided MMIF method that integrates feature restoration and interaction. We first introduce "Pseudo Ground Truth" to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask-guided cross-modal cross-attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the "Pseudo Ground Truth". Additionally, we propose a mask-guided learning strategy and a task-coupled degradation-aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG-Fuse.
