Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical reinforcement learning framework for dexterous grasping, decoupling task-space planning via multi-agent RL from joint-space control using quadratic programming. This approach accelerates training, enforces hardware safety, and enables zero-shot steerability for adjusting safety margins and avoiding obstacles. Real-world experiments on a 7-DoF arm with a 20-DoF hand demonstrate robust zero-shot transferability and reactive recovery from disturbances on unseen objects.
Reactive dexterous grasping can be achieved with zero-shot transfer to real-world objects by decoupling high-level RL planning from low-level QP control, enabling dynamic adjustments to safety margins without retraining.
In this work, we propose a hybrid hierarchical control framework for reactive dexterous grasping that explicitly decouples high-level spatial intent from low-level joint execution. We introduce a multi-agent reinforcement learning architecture, specialized into distinct arm and hand agents, that acts as a high-level planner by generating desired task-space velocity commands. These commands are then processed by a GPU-parallelized quadratic programming controller, which translates them into feasible joint velocities while strictly enforcing kinematic limits and collision avoidance. This structural isolation not only accelerates training convergence but also strictly enforces hardware safety. Furthermore, the architecture unlocks zero-shot steerability, allowing system operators to dynamically adjust safety margins and avoid dynamic obstacles without retraining the policy. We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline. Real-world hardware experiments on a 7-DoF arm equipped with a 20-DoF anthropomorphic hand demonstrate highly robust zero-shot transferability for dexterous grasping to a diverse set of unseen objects, highlighting the system's ability to reactively recover from unexpected physical disturbances in unstructured environments.
