Search papers, labs, and topics across Lattice.
This paper introduces Mana, a novel framework that reinterprets dexterous manipulation of articulated tools as an animation problem, leveraging a coarse-to-fine pipeline that integrates motion planning and reinforcement learning. By automating the data generation process, Mana significantly reduces the time required to specify functional affordances for various tools, achieving zero-shot sim-to-real transfer across multiple articulated tools. The results highlight Mana's scalability and effectiveness in enabling robots to grasp and manipulate complex tools with minimal human intervention.
Zero-shot sim-to-real transfer for articulated tool manipulation is now achievable with just a few clicks, revolutionizing how robots interact with complex objects.
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.
BAIR
Stanford HAI