Search papers, labs, and topics across Lattice.
This paper introduces TopoRetarget, an innovative framework for retargeting human hand-object demonstrations to robotic hands while preserving critical interaction structures. By employing a sparse interaction graph and optimizing for distance-weighted Laplacian deformation, the method ensures that task-relevant hand-object contacts are maintained, significantly enhancing the performance of reinforcement learning policies. Evaluations reveal that TopoRetarget not only achieves superior contact precision but also boosts training success rates by over 40% compared to existing methods, demonstrating its effectiveness in real-world robotic applications.
Retargeting human demonstrations to robots can now achieve a 40.6% increase in training success while preserving crucial hand-object interactions.
Human hand-object demonstrations provide dense reference motions for training dexterous manipulation reinforcement learning (RL) policies through reference tracking. However, to use such demonstrations for RL policy learning, retargeting must preserve hand pose and task-relevant hand-object contact structure. Otherwise, contact and feasibility artifacts can degrade downstream RL policy performance. We introduce TopoRetarget, an interaction-preserving retargeting framework that uses a single set of parameters across diverse retargeting conditions while maintaining task-relevant hand-object interaction and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning.
Tsinghua AI