Search papers, labs, and topics across Lattice.
This paper introduces a self-supervised method for establishing a shared canonical object frame from in-the-wild videos, eliminating the need for manual pose annotations. By leveraging noisy camera poses from Structure-from-Motion and routing training sequences through a coarse canonical mesh, the authors achieve dense correspondences between image pixels and the mesh, resulting in a common frame that captures multi-view consistency. The approach demonstrates competitive accuracy in category-level pose estimation benchmarks, outperforming traditional methods reliant on canonical pose supervision while being trained on a large dataset of 160,000 videos.
A self-supervised approach reveals that a shared canonical object frame can emerge from noisy video data, achieving pose estimation accuracy without manual annotations.
Comparing object orientations and positions across different instances requires their poses to be expressed in a shared canonical frame. Establishing such frames has traditionally required manual annotation, creating a scaling bottleneck that limits category and instance diversity. We show that a shared canonical frame can instead emerge from self-supervised training on object-centric videos captured in the wild, using only noisy camera poses from Structure-from-Motion. Our key idea is to route all training sequences through a shared geometric bottleneck: a coarse canonical mesh that carries no category-specific detail. By learning dense correspondences from image pixels to this mesh, and estimating per-sequence alignments from noisy SfM geometry, a common canonical frame emerges from multi-view consistency and the semantic priors of the feature extractor, without any canonical pose labels or category conditioning. Trained in a self-supervised manner on 160,000 in-the-wild object videos, our method achieves competitive accuracy on category-level pose estimation benchmarks compared to methods that rely on canonical pose supervision. The code and checkpoint is available on https://github.com/Fischer-Tom/Emergent-Canonical-Frame/.
