Search papers, labs, and topics across Lattice.
FutureWorlds addresses the challenge of learning from alternative future predictions in robotic world models by unifying candidate construction, history maintenance, and relative quality learning. The framework employs diverse beam search and candidate-specific bounded memory within a multimodal discrete autoregressive model to generate informative futures. Experimental results on RT-1, BridgeV2, and RoboCasa demonstrate significant reductions in LPIPS for long-horizon predictions and improvements in motion accuracy and object state consistency.
Robotic world models can now learn from diverse, yet confident, alternative futures, achieving state-of-the-art prediction quality and extending predictive horizons.
Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
