Search papers, labs, and topics across Lattice.
This paper introduces WorldDP, a novel hierarchical framework that integrates object-centric world models with a diffusion policy to tackle multi-stage robotic manipulation tasks. By employing a high-level world model for subgoal optimization and a low-level diffusion policy for execution, the approach effectively decouples environmental entities, allowing for improved sequential planning. Evaluations across various robotics benchmarks demonstrate that WorldDP significantly outperforms existing methods, showcasing the benefits of combining physically grounded planning with efficient execution strategies.
WorldDP achieves superior performance in multi-stage robotic tasks by seamlessly integrating high-level planning with low-level execution, outperforming traditional methods.
Visual world models have shown great potential in learning complex system dynamics. Recent advancements leverage these models as transition functions within Model Predictive Control (MPC) frameworks to solve various control tasks. When applied to robotics, however, they are limited to single-stage tasks such as reaching or grasping, and struggle with multi-stage ones that demand complex sequential planning. In this work, we introduce WorldDP, a world model framework designed for multi-stage robotic manipulation. Our hierarchical approach utilizes a high-level world model as a transition function to optimize for feasible subgoals during runtime, which are subsequently reached by a low-level Diffusion Policy. To further aid in learning dynamics and planning, we incorporate object-centric representations that decouple environmental entities and enable us to plan sequentially with respect to each. Evaluated across several robotics benchmarks, WorldDP consistently outperforms existing baselines, validating that coupling the world model's physically grounded planning with diffusion policy's efficient execution yields superior multi-stage performance.
Meta AI