Search papers, labs, and topics across Lattice.
This paper introduces BiCompoDiff, a novel compositional diffusion and energy-based framework for bimanual object reorientation that optimally integrates grasp selection, handover, regrasp, and motion planning while addressing multiple constraints. The method employs a pretrained grasp diffusion model alongside energy-based models to ensure collision avoidance and trajectory smoothness, significantly enhancing performance in complex reorientation tasks. Experimental results show that BiCompoDiff outperforms traditional sampling-based approaches, achieving over 20% higher success rates and 37% smoother trajectories, with successful real-world application demonstrating robust sim-to-real transfer.
Achieving over 20% higher success rates in bimanual object reorientation tasks reveals the power of compositional diffusion combined with energy-based optimization.
Bimanual object reorientation - picking an object, handing it over between two arms, and placing it in a desired target pose - is valuable when direct placement from the initial grasp is infeasible due to collisions, kinematic constraints, or poor final orientation. However, achieving this under multiple competing objectives remains challenging. We introduce BiCompoDiff, a compositional diffusion and energy-based framework that jointly optimizes grasp selection, handover, regrasp, and motion planning under multiple constraints. By combining a pretrained grasp diffusion model with bimanual planning energy-based models (EBMs), our method injects gradient guidance during reverse diffusion to enforce collision avoidance, trajectory smoothness (via differentiable inverse kinematics), handover feasibility, and regrasp safety. Annealed MCMC sampling further refines grasp poses over the composite energy landscape. Experiments across diverse simulated household reorientation tasks demonstrate that BiCompoDiff achieves over 20% higher success rates and up to 37% smoother trajectories (measured by joint displacement) compared to strong sampling-based baselines. Real-world validation confirms effective sim-to-real transfer and robust performance on challenging scenes.
