Search papers, labs, and topics across Lattice.
Tsinghua AIBeihangCUHKHKUSTIndependent ResearcherPengcheng LaboratorySJTUSYSUTencent AIVJun 25, 2026arXiv:2606.26694This paper introduces PhysEditWorld, a large-scale multimodal dataset designed to enhance the controllability of physics in game world models by explicitly incorporating physical parameters, particularly gravity. By leveraging a replay-and-rendering pipeline in Unreal Engine 5, the dataset captures over 100 hours of gameplay interactions across 12 cinematic scenes, providing synchronized multimodal signals that facilitate the study of physical dynamics. Initial utility studies reveal that PhysEditWorld significantly improves gravity-faithful dynamics modeling and enhances consistency when applying physical edits, paving the way for more sophisticated world modeling research.
PhysEditWorld reveals that explicit control over physical parameters can transform how game world models interact with their environments, leading to more realistic and manipulable simulations.
Recent game world models can synthesize visually plausible, action-conditioned rollouts. However, their interaction behaviors often remain limited to exploratory or wandering trajectories, and physical dynamics are typically learned as implicit correlations from data rather than as controllable variables. This limitation hinders their applicability to authored game environments, where physical rules are deliberately designed and require explicit manipulation. We introduce PhysEditWorld, a multimodal dataset with physical parameters, with a primary focus on gravity in this initial version. At its core, PhysEditWorld is built upon a replay paradigm implemented with a UE5 replay-and-rendering pipeline. Each scenario records a normalized action trace and replays the same initial state, character controller, action sequence, and camera policy under multiple gravity configurations, enabling controlled and attributable physical variation. PhysEditWorld contains 12 cinematic UE5 scenes, over 100 hours of gameplay interactions, and more than 60 million rendered rollout frames. Each sample provides synchronized multimodal signals, including RGB, depth, normals, audio, action traces, camera trajectory, engine states, semantic annotations, and explicit gravity labels. We further conduct initial utility studies on both generative video models and world understanding models, demonstrating that PhysEditWorld enables improved gravity-faithful dynamics modeling, enhances consistency under physical edits, and provides a scalable foundation for controllable world modeling research.
