Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study presents a system enabling the semi-humanoid robot NICO to imitate human arm gestures by leveraging analytical geometry and a pretrained MediaPipe pose-estimation model. By extracting 3D coordinates of human body landmarks from RGB frames and calculating joint angles through geometric relations, NICO can effectively replicate arm movements in real-time. Preliminary experiments reveal that while the method successfully captures basic gestures, it struggles with more complex poses and wrist movements, underscoring areas for future improvement.
NICO can imitate human arm gestures from simple RGB input, but struggles with complex poses, revealing critical limitations in current HRI capabilities.
Seamless human-robot interaction (HRI) requires a number of perceptual and motor abilities from the robot, one of them being the imitation of human gestures. Humanoid robots have an advantage in HRI thanks to their anthropomorphic features. In this work, we develop a system for imitation of human arm gestures by the semi-humanoid robot NICO based on analytical geometry and a pretrained MediaPipe pose-estimation model. For each input RGB frame, 3D coordinates of relevant human body landmarks, including arm joints and hand keypoints, are obtained using the MediaPipe framework. Joint angles are then computed from these coordinates using derived geometric relations. Finally, the computed angles are properly mapped to NICO's motor configuration and executed in a predefined motion sequence. Preliminary experiments on several representative arm gestures with six participants of different height indicate that the proposed method can produce meaningful imitative motions from monocular RGB input only, while also highlighting limitations in more complex poses and wrist-related movements.