Search papers, labs, and topics across Lattice.
ReMAP-DP is introduced to enhance robot policies by integrating 3D spatial awareness with 2D visual representations using a dual-stream diffusion policy. The framework employs standardized perspective reprojection coupled with pixel-aligned PointMaps to fuse semantic features and geometric descriptors. Experiments on RoboTwin 2.0 and ManiSkill 3 show ReMAP-DP outperforms the DP3 baseline, demonstrating improved performance in manipulation tasks and real-world robustness with high data efficiency.
Robots get a spatial reasoning boost: ReMAP-DP fuses multi-view images and point clouds in a diffusion policy, achieving state-of-the-art manipulation performance with surprisingly little data.
Generalist robot policies built upon 2D visual representations excel at semantic reasoning but inherently lack the explicit 3D spatial awareness required for high-precision tasks. Existing 3D integration methods struggle to bridge this gap due to the structural irregularity of sparse point clouds and the geometric distortion introduced by multi-view orthographic rendering. To overcome these barriers, we present ReMAP-DP, a novel framework synergizing standardized perspective reprojection with a structure-aware dual-stream diffusion policy. By coupling the re-projected views with pixel-aligned PointMaps, our dual-stream architecture leverages learnable modality embeddings to fuse frozen semantic features and explicit geometric descriptors, ensuring precise implicit patch-level alignment. Extensive experiments across simulation and real-world environments demonstrate ReMAP-DP's superior performance in diverse manipulation tasks. On RoboTwin 2.0, it attains a 59.3% average success rate, outperforming the DP3 baseline by +6.6%. On ManiSkill 3, our method yields a 28% improvement over DP3 on the geometrically challenging Stack Cube task. Furthermore, ReMAP-DP exhibits remarkable real-world robustness, executing high-precision and dynamic manipulations with superior data efficiency from only a handful of demonstrations. Project page is available at: https://icr-lab.github.io/ReMAP-DP/
