Search papers, labs, and topics across Lattice.
This paper introduces Ms.Forcing, a novel streaming video generation paradigm that optimizes the denoising process by adapting spatial granularity to the noise level of each state. By implementing Multi-Scale Patchification (MSP) and Multi-Scale Self-Attention (MSSA), the method reduces the active-window token count by 45% and achieves a processing speed of 22.84 FPS on a single H200 GPU, outperforming previous rolling-window methods. The approach not only enhances throughput and stability in video generation but also improves performance metrics in both short and long video settings.
Ms.Forcing achieves a 39.6% speedup in streaming video generation while significantly enhancing quality by intelligently adapting spatial granularity to noise levels.
Streaming video diffusion models have made substantial progress toward interactive and dynamic world simulation, but the nested autoregressive and denoising loops of conventional next-frame generation hinder real-time deployment. Recent rolling-window methods pipeline denoising across multiple consecutive frames at different noise levels, improving throughput and long-horizon stability. However, they tokenize every state at the same fine spatial granularity, leaving substantial noise-dependent redundancy in the joint denoising window. We propose Ms.Forcing, an efficient streaming video generation paradigm that adapts spatial granularity to each state's noise level. Its Multi-Scale Patchification (MSP) assigns coarser patches to noisier states, reducing the active-window token count by 45%, while Multi-Scale Self-Attention (MSSA) matches the density of visible non-sink keys and values to each query scale to further reduce attention cost. Because both schedules are fixed by window position, Ms.Forcing retains a static, hardware-friendly computation graph. We further introduce Homogeneous-Noise-Level DMD (H-DMD), which assembles each fake video from clean predictions sharing the same source noise level, thereby reducing the mismatch between DMD training sequences and inference-time rollouts. The multi-scale design helps offset the additional training cost of backpropagating through overlapping windows. We include both quantitative and qualitative experiments to show that Ms.Forcing reaches 22.84 FPS on a single H200 GPU, 39.6% faster than Rolling Forcing, while significantly improving VBench scores in both short video and long video generation setting.
MIT CSAIL