Search papers, labs, and topics across Lattice.
This paper introduces HandEdit, a comprehensive dataset and benchmark designed to facilitate the transformation of human hand images into robotic embodiments for dexterous manipulation. By leveraging over 200 million editing instances from diverse sources, HandEdit addresses the significant challenges posed by the discrepancies in appearance and articulation between human and robotic data. The extensive evaluation of 11 image-editing baselines reveals that HandEdit not only enhances embodiment-aware editing models but also enables scalable learning for dexterous robotics from egocentric human video data.
Bridging the gap between human and robotic manipulation, HandEdit enables scalable learning for dexterous robotics using abundant egocentric video data.
Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware teleoperation data. While abundant egocentric videos of human hands offer a scalable alternative, the profound discrepancies in appearance, articulation, and camera viewpoints between human and robotic data raise significant challenges for co-training. Though existing general image-editing models demonstrate strong capabilities, they lack necessary embodiment-specific priors to fully bridge this gap. In this work, we present HandEdit, a unified large-scale embodiment-aware image-editing dataset and benchmark specifically designed to transform human hands and arms into various dexterous robotic embodiments within egocentric frames. HandEdit comprises over 200M editing instances derived from five diverse source datasets, covering 26 distinct URDFs, including 13 hand-only and 13 hand-arm configurations. Alongside the dataset, we establish a unified benchmark protocol with two tracks: Hand-only and Hand-Arm, supporting URDF-conditioned evaluation. We conduct extensive evaluations of 11 representative image-editing baselines using a multi-dimensional metric suite, including generic similarity metrics, VLM-based judgment, and embodiment-aware metrics. HandEdit serves as a critical resource at the intersection of image editing and robotics: it advances embodiment-aware editing models while enabling scalable dexterous robotic learning from abundant human video data, paving the way for more generalizable Embodied AI.
