Search papers, labs, and topics across Lattice.
The paper introduces AdaWorldPolicy, a framework for robotic manipulation that integrates a world model, action expert, and force predictor, all implemented as interconnected Flow Matching Diffusion Transformers (DiT). It leverages world models for online adaptive learning in dynamic environments, enhanced by force-torque feedback. The proposed Online Adaptive Learning (AdaOL) strategy dynamically switches between action generation and future imagination to drive reactive updates, enabling adaptation to visual and physical domain shifts, achieving state-of-the-art performance in simulated and real-robot benchmarks.
Robots can now adapt to dynamic environments with minimal human involvement by learning from a world model and force-torque feedback, achieving state-of-the-art manipulation performance.
Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. In this work, we introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy) to enhance robotic manipulation under dynamic conditions with minimal human involvement. Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments, which can be complemented by force-torque feedback to mitigate dynamic force shifts. Our AdaWorldPolicy integrates a world model, an action expert, and a force predictor-all implemented as interconnected Flow Matching Diffusion Transformers (DiT). They are interconnected via the multi-modal self-attention layers, enabling deep feature exchange for joint learning while preserving their distinct modularity characteristics. We further propose a novel Online Adaptive Learning (AdaOL) strategy that dynamically switches between an Action Generation mode and a Future Imagination mode to drive reactive updates across all three modules. This creates a powerful closed-loop mechanism that adapts to both visual and physical domain shifts with minimal overhead. Across a suite of simulated and real-robot benchmarks, our AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.
