Search papers, labs, and topics across Lattice.
This paper introduces the World Value Model (WVM), which integrates world models with value estimation to enhance robotic policy learning from diverse data sources. By leveraging the temporal modeling strengths of world models, WVM achieves state-of-the-art Value-Order Correlation (VOC) results on standard benchmarks and demonstrates robustness on the newly introduced Suboptimal-Value-Bench, which includes 800 suboptimal trajectories. The findings indicate that WVM significantly improves manipulation performance in both simulated and real-world scenarios, showcasing its effectiveness in learning from mixed-quality data.
WVM outperforms existing models by accurately assessing task progressions and improving robotic manipulation from both expert and suboptimal data.
Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.
Tsinghua AI