Search papers, labs, and topics across Lattice.
This paper introduces ExS2D, a hierarchical action expansion framework that enables dual-arm manipulation using only single-arm demonstrations, significantly reducing the need for costly bimanual training data. By generating structured subtasks from textual instructions and utilizing a multimodal large language model for action allocation and planning, ExS2D achieves a 54.4% reduction in execution steps while maintaining a success rate comparable to single-arm methods. The framework's effectiveness is validated through both simulation and real-robot experiments, showcasing its reliability in dual-arm tasks with minimal prior data.
ExS2D slashes execution steps by over half while relying solely on single-arm training data, revolutionizing dual-arm manipulation efficiency.
Dual-arm manipulation can improve throughput via parallel execution, but collecting bimanual demonstrations for training is costly and difficult. We present ExS2D, a hierarchical action expansion framework that enables dual-arm manipulation from single-arm supervision. ExS2D first generates structured subtasks from textual instructions while explicitly capturing temporal precedence. It then grounds each subtask into executable actions through subtask-guided action mapping in observation. Finally, precedence-aware action allocation and synchronized planning are performed by a multimodal large language model driven coordinator to select collision-free dual-arm executions. Simulation experiments demonstrate that ExS2D reduces the average execution steps by 54.4% while maintaining a comparable success rate to a single-arm baseline. Real-robot experiments on four tasks further demonstrate the reliability of ExS2D for dual-arm execution under few-shot single-arm samples, while using zero bimanual demonstrations.
