Search papers, labs, and topics across Lattice.
PlayWorld is introduced as a fully autonomous pipeline for training high-fidelity video world simulators by leveraging unsupervised robot self-play. This approach addresses the limitations of existing video models trained on human-collected data, which often struggle with physically consistent robot-object interactions. Results demonstrate that PlayWorld generates higher quality predictions, enables fine-grained failure prediction and policy evaluation, and improves policy performance by 65% in real-world reinforcement learning tasks.
Robots can now learn better world models through unsupervised self-play, outperforming models trained on human data by 40% in failure prediction and 65% in real-world RL.
Action-conditioned video models offer a promising path to building general-purpose robot simulators that can improve directly from data. Yet, despite training on large-scale robot datasets, current state-of-the-art video models still struggle to predict physically consistent robot-object interactions that are crucial in robotic manipulation. To close this gap, we present PlayWorld, a simple, scalable, and fully autonomous pipeline for training high-fidelity video world simulators from interaction experience. In contrast to prior approaches that rely on success-biased human demonstrations, PlayWorld is the first system capable of learning entirely from unsupervised robot self-play, enabling naturally scalable data collection while capturing complex, long-tailed physical interactions essential for modeling realistic object dynamics. Experiments across diverse manipulation tasks show that PlayWorld generates high-quality, physically consistent predictions for contact-rich interactions that are not captured by world models trained on human-collected data.We further demonstrate the versatility of PlayWorld in enabling fine-grained failure prediction and policy evaluation, with up to 40% improvements over human-collected data. Finally, we demonstrate how PlayWorld enables reinforcement learning in the world model, improving policy performance by 65% in success rates when deployed in the real world.
