Search papers, labs, and topics across Lattice.
This paper introduces Spectral Lookahead Rectification (SpecLoR), an innovative method for improving text-to-video generation by addressing the drift and noise in latent ODE sampling. By leveraging lookahead prediction and shifting corrections to the frequency domain, SpecLoR effectively rectifies spatiotemporal inconsistencies without disrupting local geometry or incurring significant computational costs. Experiments show that SpecLoR significantly enhances motion coherence and reduces artifacts in generated videos, achieving these improvements with only a minimal increase in computational overhead.
SpecLoR cuts through noise and drift in text-to-video generation, achieving remarkable motion coherence with minimal computational cost.
Flow Matching has enabled robust text-to-video generation via latent ODE sampling. However, velocity approximation and numerical discretization errors inevitably accumulate, causing sampling trajectories to drift. Consequently, generated videos often suffer from severe spatiotemporal inconsistencies. Nevertheless, directly correcting these drifted, noisy latents is challenging: (i) timestep-dependent noise obscures reliable structural cues; (ii) spatial interventions risk disrupting intricate local geometry while incurring heavy computational costs. To address this, we propose Spectral Lookahead Rectification (SpecLoR), a plug-and-play inference method that bypasses noise via lookahead prediction, and circumvents spatiotemporal entanglement by shifting corrections to the frequency domain, where universal statistical priors of natural videos are readily available. First, during early sampling stages, SpecLoR looks ahead to estimate the clean latent $z_{t,0}$ and computes its 3D spatiotemporal spectrum. Next, SpecLoR rectifies the amplitude spectrum to match the prior, leaving the phase intact. Finally, the corrected state is re-noised to resume ODE integration. Experiments on Wan2.2 demonstrate that SpecLoR significantly reduces physical artifacts and enhances motion coherence across multiple benchmarks with minimal computational overhead (4 additional NFEs).
