Search papers, labs, and topics across Lattice.
The paper introduces Motus2, a self-evolving general world model designed for dexterous manipulation, which integrates a policy, simulator, and evaluator into a closed-loop decision-and-learning system. By leveraging curated expert demonstrations and learning from both successful and failed interactions, Motus2 enhances its dynamics modeling and value learning capabilities. The approach employs large-scale monocular and synchronized stereo egocentric data, along with robot-domain adaptation, to create a robust framework for improving manipulation tasks in a biomimetic robotic platform.
A closed-loop world model that learns from both successes and failures could revolutionize dexterous manipulation in robotics.
General embodied agents should perceive, predict, act, evaluate, and improve within a unified system. World models have shown great promise in building such agents, yet existing models typically append an action output head to a world simulator, without coupling them into a closed decision-and-learning loop for policy improvement. We present Motus2, a self-evolving general world model for dexterous manipulation. Motus2 advances world modeling through model scaling and data scaling. For model scaling, a single model with shared weights exposes three control interfaces: a policy (world-action model), a simulator (action-conditioned world model), and an evaluator (value model). The policy proposes candidate action chunks, the simulator predicts their visual consequences, and the evaluator assesses the predicted outcomes. Their coupling forms a closed decision-and-learning loop for policy improvement. This formulation uses curated expert demonstrations for action learning, while failed and suboptimal interactions provide valuable evidence for dynamics modeling and value learning. For data scaling, Motus2 progresses from large-scale monocular egocentric data to synchronized stereo egocentric data, followed by robot-domain adaptation with robot trajectories and supplementary human-robot alignment data. Motus2 further studies global-autoregressive and hybrid-memory extensions of its sliding-window context, adds tactile feedback for contact-aware control, and is instantiated on a fully biomimetic platform with stereo vision, dual arms, dual dexterous hands, and tactile sensing. Together, egocentric data scaling and closed-loop general world model scaling provide a general path toward self-evolving dexterous manipulation.
Tsinghua AI