Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates where to best incorporate rotation invariance in modern sparse feature matching pipelines. Through extensive experiments on 3D vision datasets, the authors find that learning rotation invariance within the feature descriptor achieves comparable performance to handling it in the matcher itself. Critically, descriptor-level invariance allows for a faster, rotation-invariant matcher without sacrificing upright performance, and scales well with larger training datasets.
Surprisingly, learning rotation invariance directly in the feature descriptor yields similar performance to handling it in the matcher, but with improved speed and no loss in upright accuracy.
Finding matching keypoints between images is a core problem in 3D computer vision. However, modern matchers struggle with large in-plane rotations. A straightforward mitigation is to learn rotation invariance via data augmentation. However, it remains unclear at which stage rotation invariance should be incorporated. In this paper, we study this in the context of a modern sparse matching pipeline. We perform extensive experiments by training on a large collection of 3D vision datasets and evaluating on popular image matching benchmarks. Surprisingly, we find that incorporating rotation invariance already in the descriptor yields similar performance to handling it in the matcher. However, rotation invariance is achieved earlier in the matcher when it is learned in the descriptor, allowing for a faster rotation-invariant matcher. Further, we find that enforcing rotation invariance does not hurt upright performance when trained at scale. Finally, we study the emergence of rotation invariance through scale and find that increasing the training data size substantially improves generalization to rotated images. We release two matchers robust to in-plane rotations that achieve state-of-the-art performance on e.g. multi-modal (WxBS), extreme (HardMatch), and satellite image matching (SatAst). Code is available at https://github.com/davnords/loma.