Search papers, labs, and topics across Lattice.
This paper introduces a novel framework called LAMP, which integrates a latent motion prior module to enhance real-world learning for dexterous hand manipulation. By mapping hand-action histories to a compact latent space, LAMP reduces the inefficiencies and risks associated with traditional reinforcement learning in high-dimensional action spaces. The approach significantly improves task success rates, achieving up to 100% success on three out of four real-robot manipulation tasks after online reinforcement learning, demonstrating its effectiveness in practical applications.
Achieving a staggering 98.75% success rate in dexterous manipulation tasks, LAMP redefines how we approach real-world learning in robotics.
Real-world learning for dexterous hands remains brittle because high-dimensional hand actions amplify imitation errors and make reinforcement-learning exploration prone to contact-breaking motion. While combining imitation learning (IL) with online reinforcement learning (RL) can reduce manual supervision, unconstrained exploration in raw hand-action spaces is sample-inefficient and risky for physical hardware. We introduce a latent motion prior module (\prior{}) that maps recent hand-action histories to a compact, history-conditioned latent prior and decodes continuous latent commands into executable high-dimensional hand targets. Built on this prior, \method{} is a three-stage real-world dexterous learning framework: it pretrains \prior{} from demonstrations, trains a visuomotor policy that predicts native arm commands and latent hand-action offsets, and improves the policy with online residual RL in the same latent hand-action space. This shared, decodable interface lets residual exploration make local corrections near demonstrated, contact-consistent hand motions rather than perturbing every finger joint independently. We evaluate \method{} on four real-robot dexterous manipulation tasks against raw, linear, and discrete hand-action interfaces. Starting from small task-specific demonstration sets, \method{} achieves a 56.25\% average IL success rate and raises it to 98.75\% after online RL, reaching 100\% final success on three tasks and 95\% on the remaining task.
Tsinghua AI