Search papers, labs, and topics across Lattice.
This paper introduces Object-centric LeJEPA, an extension of the LeJEPA framework that aligns object-centric representations using off-the-shelf object masks from SAM proposals, addressing the instability inherent in self-supervised object representation. By leveraging a distributional anti-collapse objective and incorporating an instance-separating loss, the method improves data efficiency and performance on downstream tasks. The results show that Object-centric LeJEPA outperforms traditional image-level LeJEPA across multiple benchmarks, demonstrating its effectiveness in tasks such as tracking, classification, segmentation, and re-identification with significantly reduced training data.
Object-centric LeJEPA achieves superior performance on key vision tasks while requiring significantly less training data than traditional image-level methods.
Image encoders trained with LeJEPA can deliver strong features for downstream tasks, but, like other image-level self-supervised methods, typically require large training datasets. Aligning representations at the level of objects rather than whole scenes promises greater data efficiency, but doing this in a completely self-supervised way, effectively jointly partitioning a scene and representing its objects, is unstable: the two are locked in a cyclic dependency, partitioning requires meaningful representations, while meaningful representations require consistent partitioning. We sidestep this instability by taking object masks as given during training, using cheap, off-the-shelf SAM proposals. We extend LeJEPA - whose distributional anti-collapse objective ports naturally from whole images to variable-sized sets of objects - to align object-centric representations rather than whole images. An additional instance-separating loss, which treats other objects in the same scene as negatives, further boosts downstream performance. Across two model scales and 10-100% of COCO, object-level LeJEPA outperforms image-level LeJEPA on tracking (DAVIS), classification (ImageNet-1k), segmentation (ADE20k), and re-identification (NAVI).
