Search papers, labs, and topics across Lattice.
The paper introduces RaCo, a lightweight neural network for learning robust keypoints by integrating a repeatable keypoint detector, a differentiable ranker to maximize matches with fewer keypoints, and a covariance estimator for quantifying spatial uncertainty. RaCo is trained on perspective image crops without requiring co-visible image pairs and uses extensive data augmentation to achieve rotational robustness, avoiding computationally expensive equivariant architectures. Evaluated on challenging datasets, RaCo achieves state-of-the-art performance in keypoint repeatability and two-view matching, especially under large in-plane rotations.
Forget complex architectures: RaCo achieves SOTA keypoint matching and repeatability by cleverly combining ranking and covariance estimation in a lightweight network, trained without covisible image pairs.
This paper introduces RaCo, a lightweight neural network designed to learn robust and versatile keypoints suitable for a variety of 3D computer vision tasks. The model integrates three key components: the repeatable keypoint detector, a differentiable ranker to maximize matches with a limited number of keypoints, and a covariance estimator to quantify spatial uncertainty in metric scale. Trained on perspective image crops only, RaCo operates without the need for covisible image pairs. It achieves strong rotational robustness through extensive data augmentation, even without the use of computationally expensive equivariant network architectures. The method is evaluated on several challenging datasets, where it demonstrates state-of-the-art performance in keypoint repeatability and two-view matching, particularly under large in-plane rotations. Ultimately, RaCo provides an effective and simple strategy to independently estimate keypoint ranking and metric covariance without additional labels, detecting interpretable and repeatable interest points. The code is available at https://github.com/cvg/RaCo.
