Search papers, labs, and topics across Lattice.
OTRetarget jointly retargets robot and multi-object motion from human demonstrations by representing surface interactions via signed distances and closest surface points, then transferring these quantities using entropic optimal transport. This unified approach integrates interaction targets into a constrained inverse kinematics formulation that optimizes robot and object poses frame-wise, preserving contact fidelity across differing geometries without scene rescaling. The method achieves significantly higher robot-object interaction Jaccard scores and lower depth errors than prior work, enabling successful transfer to physical humanoid robots for complex loco-manipulation tasks.
Humanoid robots can now seamlessly perform complex pick-and-place tasks by jointly retargeting their own motion and the manipulated objects' trajectories, achieving near-perfect interaction fidelity.
Transferring human motion to humanoid robots requires adapting the demonstrated motion to the robot morphology while preserving interactions with the environment. This is particularly challenging for loco-manipulation tasks, where contacts with the ground and manipulated objects must remain consistent despite differences in body proportions. Yet, skeletal motion alone does not fully describe these interactions, and fixing object trajectories limits the adaptation to a new embodiment. In this paper, we introduce OTR ETARGET, a unified approach to jointly retarget robot and multi-object motion from human demonstrations. Our approach represents surface interactions through signed distances, closest surface points, and relative directions, and uses entropic optimal transport to transfer these quantities across human, robot, and object geometries. We incorporate the resulting interaction targets into a constrained inverse kinematics formulation that balances contact preservation with motion style and jointly optimizes robot and object poses at each frame. This formulation accommodates robot-object and object-object interactions without rescaling the scene or the demonstration. We validate the proposed approach on OMOMO, where it achieves a robot- object interaction Jaccard score of 87% and a depth error of 8.7 mm, compared with 28% and 29.3 mm for OmniRetarget. Finally, we demonstrate transfer to a physical G1 humanoid using whole-body policies trained with reinforcement learning on the retargeted references, across motions including two-handed box pick-and-place onto a table.
