Search papers, labs, and topics across Lattice.
This study introduces AdaPT, an Adaptive Motion Planning and Tracking framework that learns professional tennis styles from broadcast videos, addressing the challenge of achieving high-performance motion in humanoid robots. The hierarchical design separates the generation of stylistic kinematic motions from their execution, enhancing task performance while minimizing interference. Real-world experiments show that the adaptation mechanism significantly improves tracking robustness and successfully bridges the sim-to-real gap, enabling effective tennis serving on humanoid robots in uncontrolled environments.
Real-world tennis serving by humanoid robots is now possible without motion capture, thanks to a novel adaptive framework that learns directly from video.
Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we propose AdaPT, an Adaptive Motion Planning and Tracking framework that learns professional tennis serving and rally styles directly from broadcast videos. This hierarchical design is motivated by the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal interference with planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades on real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perception. To address these issues, our adaptation mechanism improves tracking robustness by learning to track randomized execution speeds, while conditioning the planner on a learned motion-speed adapter to mitigate compounding errors. Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism in bridging the sim-to-real gap. We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot (1.7m) and demonstrate in-the-wild serving without motion capture. Beyond these results, our real-world experiments reveal both algorithmic and engineering insights for future humanoid ball-sports systems. Videos and code are available on our \href{https://humanoidtennis.github.io/AdaPT/}{project website}.
