Search papers, labs, and topics across Lattice.
The paper introduces STSF-Net, a novel framework for multimodal change detection (MMCD) between optical and SAR images that jointly models modality-specific and spatio-temporal common features. A key innovation is the adaptive fusion strategy guided by semantic priors from pre-trained foundational models, enabling more precise detection of semantic changes. Experiments on three datasets, including a newly introduced Delta-SN6 dataset, demonstrate STSF-Net's superior performance, outperforming state-of-the-art methods by up to 3.21% in mIoU.
Pre-trained foundational models can guide the fusion of optical and SAR features to substantially improve multimodal change detection, achieving state-of-the-art results.
Multimodal change detection (MMCD) identifies changed areas in multimodal remote sensing (RS) data, demonstrating significant application value in land use monitoring, disaster assessment, and urban sustainable development. However, literature MMCD approaches exhibit limitations in cross-modal interaction and exploiting modality-specific characteristics. This leads to insufficient modeling of fine-grained change information, thus hindering the precise detection of semantic changes in multimodal data. To address the above problems, we propose STSF-Net, a framework designed for MMCD between optical and SAR images. STSF-Net jointly models modality-specific and spatio-temporal common features to enhance change representations. Specifically, modality-specific features are exploited to capture genuine semantic change signals, while spatio-temporal common features are embedded to suppress pseudo-changes caused by differences in imaging mechanisms. Furthermore, we introduce an optical and SAR feature fusion strategy that adaptively adjusts feature importance based on semantic priors obtained from pre-trained foundational models, enabling semantic-guided adaptive fusion of multi-modal information. In addition, we introduce the Delta-SN6 dataset, the first openly-accessible multiclass MMCD benchmark consisting of very-high-resolution (VHR) fully polarimetric SAR and optical images. Experimental results on Delta-SN6, BRIGHT, and Wuhan-Het datasets demonstrate that our method outperforms the state-of-the-art (SOTA) by 3.21%, 1.08%, and 1.32% in mIoU, respectively. The associated code and Delta-SN6 dataset will be released at: https://github.com/liuxuanguang/STSF-Net.
