Search papers, labs, and topics across Lattice.
This paper introduces NestDex, a nested policy-learning framework designed to enhance dexterous manipulation in robotics by leveraging learned hand skills to simplify the demonstration collection process. By allowing operators to control arm motion while regulating hand skills through a single degree-of-freedom clutch, the framework significantly improves the efficiency and reliability of demonstrations. Empirical evaluations show that the resulting outer visuomotor policy effectively manages both arm and hand movements, leading to better performance in real-world dexterous tasks.
NestDex transforms how robots learn dexterous manipulation by enabling operators to focus on arm control while the system autonomously manages intricate finger movements.
Dexterous manipulation promises substantially richer robot interaction with the physical world, but learning these behaviours remains constrained by the difficulty of collecting consistent, complete-task demonstrations. Unlike parallel-jaw manipulation, dexterous tasks require the operator to coordinate arm motion with precise, contact-rich finger behaviour throughout the task. We introduce NestDex, a nested policy-learning framework that reduces this burden by using learned hand skills to assist demonstration collection. The operator controls the arm and regulates the active hand skill through a single-DoF clutch, rather than directly specifying the full finger trajectory. The inner hand policy adapts its motion from the latest proprioceptive history, while a vision-language selector activates the appropriate skill for each task stage. The resulting demonstrations train a separate outer visuomotor policy that controls both the arm and hand without the inner policies at deployment. A hand-action variational autoencoder provides compact hand-action targets while retaining arm commands in joint space. Across real-world dexterous manipulation experiments, NestDex improves demonstration reliability and efficiency, and the resulting empirical evaluations support effective autonomous policy learning. Video Demo are available at project website https://aus.bot/research/nestdex.
