Search papers, labs, and topics across Lattice.
This paper introduces PAWBench, a benchmark designed to evaluate video generation models based on their ability to produce probabilistically aligned world models, which capture the distribution of possible behaviors rather than just plausible trajectories. The authors find that current state-of-the-art models fail to consistently match the reference probabilities across various scenarios, indicating a significant gap in achieving probabilistic alignment. By exploring adjustments in language prompts, noise sampling, and model training, the study sets the stage for future advancements in aligning video generation with realistic world dynamics.
Current video generation models fall short of capturing the full distribution of possible behaviors, revealing a critical gap in probabilistic alignment that needs to be addressed.
Recent video generation models are increasingly framed as world models. Many physical processes can unfold in more than one valid way. Therefore, a world model should reproduce not only a plausible trajectory, but also the distribution of possible behaviors under the same initial observation and action. We call this distribution-level requirement probabilistic alignment. However, existing evaluations largely assess individual-video plausibility and do not test whether repeated generations recover the correct distribution. This raises a central question: how far are current video generators from probabilistically aligned world modeling? To answer it, we formalize probabilistic alignment as a distributional criterion for world models and introduce PAWBench, a benchmark for evaluating video generators as stochastic samplers of world dynamics. We further introduce PAWEval, an outcome-level protocol that converts repeated video rollouts into empirical distributions over possible physical behaviors. Across 50 scenarios and eleven current systems, no model consistently matches the reference probabilities while recovering the range of valid behaviors. Having established this gap, we test whether language prompts, initial noise sampling, or model training can reshape the model's predictive distribution. We believe our work can serve as a foundation for future efforts to move towards probabilistically aligned world modeling.
