Search papers, labs, and topics across Lattice.
The paper introduces Compound Adverse Weather Mamba (CAWM-Mamba), a novel end-to-end framework for infrared-visible image fusion and compound adverse weather restoration using shared weights. CAWM-Mamba employs a Weather-Aware Preprocess Module, a Cross-modal Feature Interaction Module, and a Wavelet Space State Block with Freq-SSM to model anisotropic high-frequency degradation in the wavelet domain. Experiments on the AWMM-100K benchmark and standard fusion datasets demonstrate that CAWM-Mamba outperforms state-of-the-art methods in both compound and single-weather scenarios, also improving performance on downstream tasks like semantic segmentation and object detection.
Achieve state-of-the-art image fusion and restoration in complex adverse weather by unifying infrared-visible fusion with compound degradation removal in a single Mamba-based model.
Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at https://github.com/Feecuin/CAWM-Mamba.
Tsinghua AI