Search papers, labs, and topics across Lattice.
SceneGlue, a novel scene-aware feature matching framework, is introduced to address the limitations of traditional local feature descriptors in capturing non-local scene information for cross-view image correspondence. It employs a hybridizable matching paradigm with implicit parallel attention for global context and an explicit Visibility Transformer to categorize features into visible and invisible regions. Trained solely on local feature matches without scene-level annotations, SceneGlue demonstrates improved accuracy, robustness, and interpretability across various applications.
SceneGlue achieves state-of-the-art feature matching by cleverly synthesizing implicit attention and explicit visibility estimation, all without requiring costly scene-level annotations.
Local feature matching plays a critical role in understanding the correspondence between cross-view images. However, traditional methods are constrained by the inherent local nature of feature descriptors, limiting their ability to capture non-local scene information that is essential for accurate cross-view correspondence. In this paper, we introduce SceneGlue, a scene-aware feature matching framework designed to overcome these limitations. SceneGlue leverages a hybridizable matching paradigm that integrates implicit parallel attention and explicit cross-view visibility estimation. The parallel attention mechanism simultaneously exchanges information among local descriptors within and across images, enhancing the scene's global context. To further enrich the scene awareness, we propose the Visibility Transformer, which explicitly categorizes features into visible and invisible regions, providing an understanding of cross-view scene visibility. By combining explicit and implicit scene-level awareness, SceneGlue effectively compensates for the local descriptor constraints. Notably, SceneGlue is trained using only local feature matches, without requiring scene-level groundtruth annotations. This scene-aware approach not only improves accuracy and robustness but also enhances interpretability compared to traditional methods. Extensive experiments on applications such as homography estimation, pose estimation, image matching, and visual localization validate SceneGlue's superior performance. The source code is available at https://github.com/songlin-du/SceneGlue.
