Search papers, labs, and topics across Lattice.
This paper introduces LangSCD, a vision-language framework for scene change detection that leverages VLMs to generate textual descriptions of scene changes and fuses them with visual features. A geometric-semantic matching module refines change masks by enforcing semantic consistency and spatial completeness. The authors also contribute NYC-CD, a new large-scale dataset with multiclass change annotations, and demonstrate state-of-the-art performance on multiple street-view benchmarks.
Scene change detection gets a semantic boost: fusing vision with language unlocks state-of-the-art performance, even in complex urban environments.
Scene change detection (SCD) is crucial for urban monitoring and navigation but remains challenging in real-world environments due to lighting variations, seasonal shifts, viewpoint differences, and complex urban layouts. Existing methods rely primarily on low-level visual features, limiting their ability to accurately identify changed objects amid the visual complexity of urban scenes. In this paper, we propose LangSCD, a vision-language framework for scene change detection that overcomes this single-modal limitation by incorporating semantic reasoning through language. Our approach introduces a modular language component that leverages vision-language models (VLMs) to generate textual descriptions of scene changes, which are fused with visual features through a cross-modal feature enhancer. We further introduce a geometric-semantic matching module that refines the predicted masks by enforcing semantic consistency and spatial completeness. Existing real-world scene change detection benchmarks provide only binary change annotations, which are insufficient for downstream applications requiring fine-grained understanding of scene dynamics. To address this limitation, we introduce NYC-CD, a large-scale dataset of 8,122 real-world image pairs collected in New York City with multiclass change annotations generated through a semi-automatic pipeline. Extensive experiments across multiple street-view benchmarks demonstrate that our language and matching modules consistently improve existing change-detection architectures, achieving state-of-the-art performance and highlighting the value of integrating linguistic reasoning with visual representations for robust scene change detection.
