Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper investigates the impact of extrinsic camera pose reliability on feed-forward 3D Gaussian Splatting (3DGS) for robot manipulation, finding that pose conditioning improves novel-view fidelity but its geometric benefits are contingent on extrinsic accuracy. To address this, a calibration-aware pipeline is proposed to anchor reconstructed 3DGS scenes to the robot's metric workspace, enhancing geometric consistency. The resulting 3DROID dataset provides a renderable, metric-pose-anchored collection with per-scene reliability metrics, enabling more robust 3D supervision for robot manipulation tasks.
Unreliable camera extrinsics cripple 3D Gaussian Splatting's geometric utility for robotics, but a calibration-aware pipeline anchors reconstructions to the robot's metric workspace, unlocking reliable 3D supervision.
Robot manipulation models primarily reason from 2D observations while acting in the 3D physical world. To bridge this gap, recent work has augmented robot data with geometric priors such as depth, point clouds, and 3D trajectories, while renderable 3D Gaussian representations provide another promising form of 3D supervision. However, 3DGS representation is designed mainly for photometric fidelity and may not preserve real-world metric scale, particularly when the supplied camera extrinsics are unreliable. We study the effect of extrinsic reliability and pose conditioning on feed-forward 3DGS, and propose a calibration-aware pipeline that anchors reconstructed scenes to the robot's metric workspace. Our experiments show that pose conditioning improves novel-view fidelity, while its geometric benefit depends on the reliability of the injected extrinsics. Using this pipeline, we present a renderable, metric-pose-anchored dataset with scene-level reliability information for robot manipulation research. Our dataset is available at https://huggingface.co/datasets/wonguen/3DROID