Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces MARCO, a novel dual-encoder architecture built upon DINOv2, designed to improve semantic correspondence generalization, especially for unseen keypoints. MARCO employs a coarse-to-fine objective for spatial precision and a self-distillation framework to extend sparse supervision, enabling dense and semantically consistent correspondences. Experiments demonstrate that MARCO achieves state-of-the-art results on SPair-71k, AP-10K, and PF-PASCAL, with significant improvements in fine-grained localization and generalization, while being significantly smaller and faster than diffusion-based methods.
Semantic correspondence models can be 3x smaller and 10x faster without sacrificing accuracy or generalization, even for unseen keypoints.
Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .