Search papers, labs, and topics across Lattice.
The paper introduces 3D-DLP, a self-supervised model that learns object-centric representations from RGB-D or voxel data by decomposing scenes into 3D latent particles that encode key attributes. This approach enables the generation of interpretable segmentation maps and allows for controllable manipulation of scene configurations. Experimental results demonstrate that using these compact representations significantly enhances robotic manipulation tasks compared to traditional methods that either lack 3D structure or rely on dense inputs.
Manipulating 3D latent particles can generate novel scene configurations, revolutionizing how we approach object-centric representation in robotics.
We introduce 3D-DLP, a self-supervised object-centric representation learning model that decomposes scene-level RGB-D or voxel observations into a set of 3D latent particles. Building on the Deep Latent Particles (DLP) framework, each particle encodes disentangled attributes, including 3D keypoint position, bounding box dimensions, and appearance features, and represents a distinct entity in the scene. The model learns interpretable per-particle segmentation maps through an end-to-end self-supervised reconstruction objective. We demonstrate on both simulated and real-world datasets that the learned latent space is interpretable and controllable: by manipulating particle positions and decoding, we can generate novel scene configurations. Furthermore, we show that leveraging these compact 3D latent particles for downstream robotic manipulation improves performance over baselines that either lack explicit 3D information or rely on memory-intensive dense 3D inputs without object-centric structure. Code and videos are available at https://eubooks3003.github.io/3d-dlp.
