Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper addresses the inefficiency of few-step autoregressive video generation by observing that not all tokens require full denoising at every step. They introduce Speculative Forcing, a framework that identifies and "freezes" tokens that have sufficiently converged early, reducing computation by skipping unnecessary denoising steps. This approach maintains generation quality while achieving significant speedups, demonstrated by a 1.5x speed increase on short-video generation and a 1.35x speed increase on long-video generation.
Autoregressive video generation can be sped up by over 1.5x without sacrificing quality by intelligently freezing tokens that have already converged.
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.