Search papers, labs, and topics across Lattice.
This paper introduces SGFormer, a structure-aware matching network designed to enhance local feature matching in photogrammetry by addressing the issue of attention divergence in traditional Transformer models. By employing a semi-dense coarse-to-fine pipeline and the innovative Triple-Structure-Attention (TSA) module, SGFormer effectively refines attention on salient structures within overlapping regions, leading to improved matching reliability in challenging environments. Experimental results demonstrate that SGFormer significantly reduces attention divergence and enhances overall matching accuracy compared to existing methods.
Attention divergence in feature matching is tackled head-on, resulting in a robust model that significantly boosts matching accuracy in complex scenes.
Local feature matching is a fundamental component of photogrammetry, enabling accurate image correspondence critical for tasks such as 3D reconstruction, stereo mapping, and visual localization. While recent detector-free matching methods, like LoFTR, have advanced the field, the global features obtained by leveraging the global-range modeling capacity of the unconstrained attention mechanism compromise the model's attention to the salient structures in certain scenarios. This limitation leads to a phenomenon we define as attention divergence, wherein a portion of high-confidence matches are distributed outside the valid matching region (overlapping region), especially in scenes with large viewpoint variations. This occurs because similar features in irrelevant regions may receive equal weighting and consideration within the standard Transformer, limiting matching reliability in challenging photogrammetric environments. To address this issue in feature matching, we propose SGFormer (Structure-Guided Transformer), a novel structure-aware matching network that adaptively updates attention on features near salient structure in overlapping regions. SGFormer employs a semi-dense coarse-to-fine pipeline and incorporates the proposed Triple-Structure-Attention (TSA) module into the backbone net for extracting distinctive features. The TSA module utilizes shallow local features from early network layers to enhance the representation around salient structure, guiding subsequent transformer stages to intensify the model's focus on regions with salient structure across the global scope. SGFormer, thereby reinforcing attention to visually consistent areas while mitigating the influence of non-overlapping regions. Extensive experiments show that SGFormer significantly mitigates attention divergence and improves matching accuracy.
