Search papers, labs, and topics across Lattice.
This paper introduces WorldRoamBench, an innovative benchmark designed to evaluate the long-horizon stability of interactive world models (IWMs) across four critical dimensions: action, vision, physics, and memory. By employing tailored metrics that address the limitations of existing benchmarks, such as per-frame action evaluation and segment-based drift metrics, the authors reveal significant shortcomings in current IWMs, with none of the evaluated models achieving satisfactory performance across all dimensions. The findings underscore the need for IWMs that are not only stable and physically grounded but also capable of maintaining memory fidelity in real-world applications.
Current interactive world models fall short, with none passing the rigorous tests of WorldRoamBench designed to assess long-horizon stability across action, vision, physics, and memory.
Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored innovations: (i) Action: per-frame action metric bypassing cross-model semantic scale disparity and exposing failures hidden by trajectory; (ii) Vision: segment-based drift metric capturing non-monotonic mid-sequence collapse missed by start-vs-end comparisons; (iii) Physics: controllability-gated evaluation over mechanics, optics, and 3D consistency, scoring plausibility under faithful action execution; (iv) Memory: action-decoupled protocol evaluating scene memory via transition-localized 3D point-cloud reconstruction and subject memory via tracking-plus-VLM reasoning. The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/third-person views with WASD 10-60s continuous interaction. Evaluating 10+ open/closed-source models reveals none reliably satisfies all dimensions; even the best achieves only moderate scores. Advances on WorldRoamBench are steps toward IWMs that are stable, physically grounded, memory-faithful, and deployable in real-world applications.
DAMO
Tsinghua AI