Search papers, labs, and topics across Lattice.
This paper introduces Adversarial Posture Regularization (APR) to enhance the realism of bimanual dexterous piano playing in reinforcement learning models by aligning hand postures with human-like distributions. By leveraging a small dataset of casual human playing rather than extensive expert demonstrations, APR effectively mitigates issues of unnatural postures and joint overextension. The results demonstrate a marked improvement in human-likeness metrics and visual quality compared to previous methods, showcasing the effectiveness of the approach in simulating realistic piano playing.
Adversarial Posture Regularization transforms piano-playing robots from awkward automatons into fluid, human-like performers using minimal human data.
Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high-DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose \textit{Adversarial Posture Regularization (APR)}. It avoids expensive, song-aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human-like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer-grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human-likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: https://github.com/APRProject/APRPianist.
Tsinghua AI