Search papers, labs, and topics across Lattice.
This paper introduces GESS, a novel framework for learning robust local features by integrating semantic and geometric cues. GESS uses a joint semantic-normal prediction head and a depth stability prediction head to guide keypoint selection and descriptor construction. The proposed Semantic-Depth Aware Keypoint (SDAK) mechanism and Unified Triple-Cue Fusion (UTCF) module significantly improve feature detection and descriptor discriminability, as validated on multiple benchmarks.
Fusing semantic and geometric cues yields more robust and discriminative local features, outperforming methods relying on single appearance cues.
Robust local feature detection and description are foundational tasks in computer vision. Existing methods primarily rely on single appearance cues for modeling, leading to unstable keypoints and insufficient descriptor discriminability. In this paper, we propose a multi-cue guided local feature learning framework that leverages semantic and geometric cues to synergistically enhance detection robustness and descriptor discriminability. Specifically, we construct a joint semantic-normal prediction head and a depth stability prediction head atop a lightweight backbone. The former leverages a shared 3D vector field to deeply couple semantic and normal cues, thereby resolving optimization interference from heterogeneous inconsistencies. The latter quantifies the reliability of local regions from a geometric consistency perspective, providing deterministic guidance for robust keypoint selection. Based on these predictions, we introduce the Semantic-Depth Aware Keypoint (SDAK) mechanism for feature detection. By coupling semantic reliability with depth stability, SDAK reweights keypoint responses to suppress spurious features in unreliable regions. For descriptor construction, we design a Unified Triple-Cue Fusion (UTCF) module, which employs a semantic-scheduled gating mechanism to adaptively inject multi-attribute features, improving descriptor discriminability. Extensive experiments on four benchmarks validate the effectiveness of the proposed framework. The source code and pre-trained model will be available at: https://github.com/yiyscut/GESS.git.
