Search papers, labs, and topics across Lattice.
This paper introduces DINOcular, a self-supervised framework that leverages RGB-D observations to learn joint visuospatial representations by integrating depth-derived geometric priors with a visual backbone. The method employs inter-patch and intra-patch fusion to efficiently encode both appearance and spatial structure, addressing the limitations of traditional RGB-only models. Key results demonstrate significant improvements in 3D awareness and competitive performance in RGB-D semantic segmentation tasks compared to existing methods of similar scale.
Integrating depth information with visual data leads to a dramatic boost in 3D awareness, outperforming traditional methods on key benchmarks.
We introduce a self-supervised framework for learning joint visuospatial representations from RGB-D observations. While modern vision foundation models are trained almost exclusively on RGB images, many embodied systems have access to explicit depth sensing, which provides geometric information that monocular inputs cannot recover. Our method integrates depth-derived geometric priors with a visual backbone through inter-patch and intra-patch fusion, enabling the model to encode both appearance and spatial structure efficiently. The resulting representation shows promising improvements on 3D awareness while preserving semantic transfer: it outperforms prior methods of comparable scale on multiple 3D geometry benchmarks, and remains competitive when probed for standard RGB-D semantic segmentation tasks.
