Search papers, labs, and topics across Lattice.
This paper introduces HANDFUL, a learning framework for sequential grasp-conditioned dexterous manipulation that explicitly models finger usage as a limited resource. By using finger-level contact rewards, HANDFUL encourages the exploration of resource-aware grasps optimized for subsequent manipulation subtasks. The approach is validated in simulation using the new HANDFUL-Bench benchmark and on a real LEAP hand, demonstrating improved success and robustness in sequential tasks compared to greedy grasp optimization.
Prioritizing resource-aware grasps, where finger usage is explicitly modeled, dramatically improves the success rate of sequential dexterous manipulation tasks.
Dexterous robot hands offer rich opportunities for multifunctional manipulation, where a robot must execute multiple skills in sequence while maintaining control over previously grasped objects. Most prior work in dexterous manipulation focuses on single-object, single-skill tasks. In contrast, our insight is that many sequential tasks require resource-aware grasps that conserve fingers for future actions. In this paper, we study sequential grasp-conditioned dexterous manipulation, where a robot first grasps an object and then performs a second, distinct manipulation subtask while preserving the initial grasp. We introduce HANDFUL, a learning framework that models finger usage as a limited resource and encourages exploration of resource-aware grasps through finger-level contact rewards. These grasps are subsequently selected for downstream tasks via curriculum-based policy learning. We further propose HANDFUL-Bench, a simulation benchmark that introduces sequential dexterous manipulation tasks across multiple secondsubtask objectives, including pushing, pulling, and pressing, under a shared grasp-conditioned setup. Extensive simulation results demonstrate that prioritizing resource-aware grasps improves second-subtask success and robustness compared to a baseline that greedily optimizes the initial grasp before attempting the second subtask. We additionally validate our approach on a real dexterous LEAP hand. Together, this work establishes resource-aware grasp planning as a key principle for multifunctional dexterous manipulation. Supplementary material is available on our website: https://handful-dex.github.io.
