Search papers, labs, and topics across Lattice.
dWorldEval introduces a scalable robotics policy evaluation method using a discrete diffusion world model that maps vision, language, and actions into a unified token space. The model employs a transformer-based denoising network with a sparse keyframe memory for spatiotemporal consistency and a progress token to indicate task completion. Experiments show dWorldEval significantly outperforms existing methods on LIBERO, RoboTwin, and real-robot tasks, demonstrating its potential for large-scale robotics evaluation.
Forget slow, expensive real-world trials: dWorldEval's discrete diffusion world model lets you evaluate robot policies across thousands of environments and tasks with unprecedented speed and accuracy.
Evaluating robotics policies across thousands of environments and thousands of tasks is infeasible with existing approaches. This motivates the need for a new methodology for scalable robotics policy evaluation. In this paper, we propose dWorldEval, which uses a discrete diffusion world model as a scalable evaluation proxy for robotics policies. Specifically, dWorldEval maps all modalities - including vision, language, and robotic actions - into a unified token space, modeling them via a single transformer-based denoising network. In this paper, we propose dWorldEval, using a discrete diffusion world model as a scalable evaluation proxy for robotics policy. Specifically, it maps all modalities, including vision, language, and robotics action into a unified token space, then denoises them with a single transformer network. Building on this architecture, we employ a sparse keyframe memory to maintain spatiotemporal consistency. We also introduce a progress token that indicates the degree of task completion. At inference, the model jointly predicts future observations and progress token, allowing automatically determine success when the progress reaches 1. Extensive experiments demonstrate that dWorldEval significantly outperforms previous approaches, i.e., WorldEval, Ctrl-World, and WorldGym, on LIBERO, RoboTwin, and multiple real-robot tasks. It paves the way for a new architectural paradigm in building world simulators for robotics evaluation at scale.
