Search papers, labs, and topics across Lattice.
This paper introduces RIPE++, an advanced reinforcement learning approach for keypoint extraction that utilizes a novel reward mechanism derived solely from positive image pairs, eliminating the need for negative samples. By leveraging geometric consistency signals, the method achieves improved training stability and descriptor discriminability, allowing for effective representation learning even in challenging scenarios with limited supervision. The authors demonstrate significant performance gains on the MegaDepth1500 benchmark and applicability to low-texture medical video sequences, showcasing its robustness compared to fully-supervised methods.
A novel reinforcement learning reward derived from positive pairs alone boosts keypoint learning stability and performance, even in low-texture environments.
Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at https://github.com/fraunhoferhhi/RIPEpp .
