Search papers, labs, and topics across Lattice.
This paper investigates the compounding error in video diffusion-based world models, revealing that the effective rank of hidden representations decreases significantly during generation drift, which contributes to frame quality degradation. The authors introduce video representation regularization as a novel training constraint that stabilizes these latent representations, effectively mitigating error accumulation during long autoregressive video generation. Their method outperforms existing approaches, achieving substantial improvements in aesthetic and imaging quality metrics on the VBench benchmark.
Compounding error in video generation is linked to representational degradation, and a simple regularization technique can dramatically enhance long-term stability.
Video diffusion-based world models enable long autoregressive video generation for robotics, autonomous driving and simulation tasks, yet sliding-window autoregressive inference suffers from severe error accumulation that degrades frame quality over time. Although this phenomenon has been widely observed, the underlying mechanism of compounding error and how to achieve stable long-horizon generation remain largely unresolved. In this paper, we investigate the internal representation dynamics of video world models and discover that compounding error is tightly coupled with dimensional collapse of hidden representations. Specifically, the effective rank of model representations sharply decreases at the onset of generation drift, revealing a strong connection between representational degradation and long-term rollout instability. Furthermore, we find that pure training data scaling fails to boost model resistance to error drift, contradicting mainstream scaling paradigms. To address this problem, we propose video representation regularization, a lightweight training constraint that stabilizes latent representations and suppresses iterative error accumulation. Compared with Diffusion Forcing, our method achieves improvements from 38.65 to 55.56 and from 44.37 to 72.08 on the Aesthetic Quality and Imaging Quality metrics of VBench. Our work establishes the first connection between autoregressive video drifting and model internal representations, adopts erank as a quantitative metric for error accumulation, reveals counterintuitive scaling limitations for video world models, and presents a simple yet effective regularization strategy to improve long video generation robustness.
