Search papers, labs, and topics across Lattice.
This paper introduces WaveSync, a hybrid framework that utilizes a Large Language Model to generate synchronized co-speech gestures for humanoid robots while adhering to strict kinematic and dynamic constraints. By decomposing dialogue into structured semantic schemas and employing Dynamic Movement Primitives, WaveSync ensures that gesture trajectories are both expressive and feasible. Experimental results demonstrate that WaveSync achieves superior synchronization accuracy compared to three baseline methods across various dialogue scenarios, highlighting the importance of integrating semantic and kinematic considerations in robotic gesture generation.
Achieving high synchronization accuracy in humanoid robot gestures requires a delicate balance of semantic understanding and kinematic feasibility, and WaveSync delivers just that.
Expressive co-speech gestures are crucial for natural human-robot interaction, but generating them on physical humanoid robots is difficult because gesture strokes must align with speech emphasis while satisfying strict kinematic and dynamic constraints. Unlike virtual avatars, humanoid robots cannot freely execute rapid or overlapping motions, making word-level synchronization and hardware-safe motion planning a coupled problem. We present \textbf{WaveSync}, a hybrid framework in which a Large Language Model decomposes dialogue responses into structured semantic schemas and assigns per-word importance weights, constructing a continuous Semantic Importance Wave. Gesture trajectories are shaped through Dynamic Movement Primitives, enforcing kinematic feasibility while enhancing expressiveness. A Wavefront Optimization stage aligns peak-to-peak gesture-speech synchronization and resolves residual kinematic violations through gesture-duration compression and forward propagation. Experimental evaluation based on five dialogue scenarios shows that our method achieves high synchronization accuracy and outperforms three baselines in both objective and subjective evaluations. Each component in WaveSync plays a necessary role in producing gestures that are expressive, semantically grounded, and kinematically compliant. The code, resources, and videos are available at \href{https://github.com/pairs-lab/WaveSync}{WaveSync}