Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Cross-Dimensional Class-Agnostic 3D Instance Segmentation (CDIS), a novel framework that enhances 3D instance segmentation by tracking 2D instance masks across frames and linking them with 3D superpoints. By creating a feedback loop between 2D and 3D representations, CDIS maintains object identities over time and produces globally consistent 3D instance labels without requiring 3D-specific training. Experimental results show that CDIS outperforms existing zero-shot methods in terms of accuracy and consistency, demonstrating its effectiveness in complex real-world scenarios.
Achieving globally consistent 3D instance labels without any 3D-specific training could revolutionize how robots perceive and interact with unknown environments.
Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments, enabling perception of previously unseen objects for reliable manipulation and navigation. Existing approaches typically project per-frame 2D instance masks into 3D and merge them, which often breaks object identities across time and yields fragmented 3D instances. We introduce Cross-Dimensional Class-Agnostic 3D Instance Segmentation (CDIS), a zero-shot framework that explicitly tracks 2D instance masks across frames and associates them with 3D superpoints, creating a feedback loop between 2D and 3D. This cross-dimensional reasoning links temporally stable 2D tracks with spatially coherent 3D regions, producing globally consistent 3D instance labels without any 3D-specific training. Experiments on benchmark datasets demonstrate that CDIS achieves higher accuracy and consistency than state-of-the-art zero-shot methods, while remaining efficient and scalable to diverse real-world environments.