Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
We introduce Speculative Sampling, a strategy for interactive video generation that leverages larger bidirectional context windows to improve video quality and GPU utilization, while maintaining low latency. This is achieved by appending noise latents as padding to expand the modeling window and caching predictions for reuse when conditions are stable, enabling incremental delivery of predictions with replaceable future frames. The method effectively decouples bidirectional window size from latency constraints, allowing for joint optimization of visual quality and sampling time without complex post-training.
Interactive video generation can now achieve high quality with large context windows without sacrificing real-time responsiveness.
Interactive video generation commonly relies on small bidirectional chunks to reduce response latency. Yet our preliminary experiments show that larger chunks improve video quality and better utilize GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time. Motivated by these observations, we propose Speculative Sampling, a strategy that combines the modeling power of large bidirectional windows with the low-latency interaction of small ones. It expands the modeling window by appending noise latents as padding and caches the resulting predictions for reuse when the condition remains unchanged. Predictions are delivered incrementally, while the uncommitted future remains replaceable. Our latency analysis shows that this removes the waiting-time lower bound imposed by chunk boundaries, shifting the optimization focus from reducing chunk size to accelerating per-chunk sampling. Bidirectional window size thus becomes a decision variable for jointly optimizing visual quality and sampling time, rather than a necessary constraint on low-latency interaction. This opens a path to serving large bidirectional models in interactive streaming systems without complex streaming post-training.