Search papers, labs, and topics across Lattice.
CyboRacket, a hierarchical framework, was developed to address the challenges of dynamic ball-interaction tasks in humanoid robots, specifically for racket sports. It integrates onboard visual perception, physics-based trajectory prediction, and pre-trained whole-body control to enable real-time interception and striking. Experiments on a Unitree G1 humanoid robot demonstrate successful vision-based tennis hitting using only onboard sensing.
Onboard perception and pre-trained whole-body control can enable humanoid robots to perform dynamic racket sports like tennis without external motion capture or task-specific retraining.
Dynamic ball-interaction tasks remain challenging for robots because they require tight perception-action coupling under limited reaction time. This challenge is especially pronounced in humanoid racket sports, where successful interception depends on accurate visual tracking, trajectory prediction, coordinated stepping, and stable whole-body striking. Existing robotic racket-sport systems often rely on external motion capture for state estimation or on task-specific low-level controllers that must be retrained across tasks and platforms. We present CyboRacket, a hierarchical perception-to-action framework for humanoid racket sports that integrates onboard visual perception, physics-based trajectory prediction, and large-scale pre-trained whole-body control. The framework uses onboard cameras to track the incoming object, predicts its future trajectory, and converts the estimated interception state into target end-effector and base-motion commands for whole-body execution by SONIC on the Unitree G1 humanoid robot. We evaluate the proposed framework in a vision-based humanoid tennis-hitting task. Experimental results demonstrate real-time visual tracking, trajectory prediction, and successful striking using purely onboard sensing.
