Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper introduces Dexterity from Supervised Play (DSP), a framework that leverages brief human play data to bootstrap robot learning of dexterous manipulation. DSP first uses behavior cloning on human play to acquire an initial policy, which then autonomously collects extensive interaction data for world model training. The learned world model subsequently guides model-predictive control and can be used to distill a closed-loop policy via self-play, achieving high success rates with significantly reduced data requirements.
Robots can learn complex dexterous manipulation from just minutes of human "play," outperforming RL by 490x in data efficiency.
How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present DSP: Dexterity from supervised play, a framework in which a robot learns a play policy from minutes of human play through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model. During inference, the same play policy then guides model-predictive control (MPC) by proposing candidate action sequences for evaluation under the learned world model. To accelerate inference, we also introduce a method for distilling a closed-loop, goal-conditioned policy through self-play within the learned world model. We evaluate this framework on underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand, in simulation and real world. In real world experiment, our method outperforms hindsight behavior cloning by 69.6% in success rate and achieves comparable success to reinforcement learning with 490 times less interaction data. These results demonstrate that supervised play can autonomously collect interaction data, guide model-based planning, and enable policy learning without any task-specific demonstrations or sim-to-real transfer.