Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We introduce HandsOn, a large-scale, composable simulation benchmark featuring diverse robotic embodiments, dexterous hands, and a wide array of objects to evaluate generalizable dexterous manipulation. The benchmark includes 66 human-verified tasks, split into demonstration-based learning (Core-50) and transfer evaluation (Transfer-16), alongside 18,000 curated human demonstrations. Current state-of-the-art multi-task policies achieve only 29.1% success on Core-50, highlighting significant challenges in contact establishment, action coordination, and goal maintenance for robust physical interaction.
Even the most advanced multi-task policies struggle to achieve 30% success on a new benchmark designed to test generalizable dexterous manipulation, revealing critical gaps in current robotic intelligence.
General-purpose manipulation requires reliable behavior across embodiments, objects, and task demands. We introduce HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation. It supports five robotic arms, six dexterous hands, and approximately 200 rigid, articulated, and deformable assets in randomized scenes. The benchmark pairs 18,000 curated human demonstrations with 66 human-verified tasks: Core-50 supports demonstration-based learning and execution tests, while Transfer-16 evaluates asset and relation transfer, skill composition, and deformable manipulation without target-task training demonstrations. Qualitative real-robot rollouts illustrate sim-to-real transfer. Across diffusion, vision-language-action, world-action, and prompted VLM-based methods, the best trained multi-task policy achieves 29.1% Core-50 success. Without benchmark finetuning, GPT-6 using move_eef for palm-and-finger control achieves 5.2% on Core-50 and 1.9% on Transfer-16 over 660 trials. Trained-policy stage diagnostics expose failures in contact establishment, action coordination, and maintaining achieved goals, with additional sequencing errors in composed tasks. HandsOn provides a shared basis for investigating manipulation generalization and the limits of reliable physical interaction.
Tsinghua AI