Search papers, labs, and topics across Lattice.
The paper introduces YoCausal, a benchmark to evaluate video diffusion models' (VDMs) causal understanding by leveraging temporally reversed real-world videos as counterfactual samples. They define the Reverse Surprise Index (RSI) to measure arrow-of-time perception and the Causality Cognition Index (CCI) using a VLM to disentangle causal reasoning from temporal bias. Experiments on 13 VDMs show that while models can perceive the arrow of time, they still lag behind human-level causal cognition.
VDMs can tell time is running backward, but still fail at basic causal reasoning, revealing a significant gap between statistical pattern matching and true world model understanding.
As video diffusion models (VDMs) advance toward world models, a key question arises: do they truly understand causality, or merely overfit to statistical temporal patterns? Existing benchmarks mostly rely on synthetic data, limiting real-world generalization due to the sim-to-real gap. We present YoCausal, a two-level benchmark inspired by the Violation of Expectation (VoE) paradigm from cognitive science. By temporally reversing real-world videos at zero cost as natural counterfactual samples, YoCausal establishes an arbitrarily extensible evaluation protocol. Level 1 introduces the Reverse Surprise Index (RSI), quantifying arrow-of-time perception via denoising loss. Level 2 introduces the Causality Cognition Index (CCI), which leverages a VLM to stratify datasets into causal and non-causal subsets, disentangling genuine causal reasoning from temporal bias. Evaluation of 13 state-of-the-art VDMs reveals that perceiving the arrow of time does not imply understanding causality, and a significant gap persists relative to human-level causal cognition.
