Search papers, labs, and topics across Lattice.
FoundObj uses reinforcement learning to train an agent to merge superpoints into object segments in 3D point clouds, without requiring manual scene-level labels. The agent is guided by semantic and geometric reward modules that leverage self-supervised 2D and 3D foundation models as priors. Experiments show FoundObj outperforms existing baselines and generalizes well to zero-shot and long-tail scenarios, demonstrating the effectiveness of foundation model priors for label-free 3D segmentation.
Foundation models can now provide effective rewards for reinforcement learning agents segmenting 3D objects, even without any manual scene-level labels.
We address the challenging task of 3D object segmentation in complex scene point clouds without relying on any scene-level human annotations during training. Existing methods are typically constrained to identifying simple objects, primarily due to insufficient object priors in the learning process. In this paper, we present FoundObj, a novel framework featuring a superpoint-based object discovery agent that incrementally merges suitable neighboring superpoints, guided by our innovative semantic and geometric reward modules. These modules synergistically leverage semantic and geometric priors from self-supervised 2D/3D foundation models, providing complementary feedback to the object discovery agent and enabling robust identification of multi-class objects through reinforcement learning. Extensive experiments on diverse benchmarks demonstrate that our approach consistently outperforms existing baselines. Notably, our method exhibits strong generalization in zero-shot and long-tail scenarios, underscoring its potential for scalable, label-free 3D object segmentation.
