Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
Autoregressive video generation struggles with joint optimization of visual quality and temporal consistency due to negative gradient alignment between frame denoising and context writing parameters. SGF+ decouples these roles by assigning separate parameters to context writing and denoising, linked via causal attention, and jointly optimizes them using the original generation objective. This approach significantly improves visual quality and long-horizon consistency, enabling continuous generation for extended durations without additional data or training time.
Decoupling denoising and context writing parameters in autoregressive video models unlocks unprecedented long-horizon generation capabilities, extending continuous output to 24 hours from just 5-second training rollouts.
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We introduce Self Gradient Forcing Plus (SGF+), which assigns separate parameters to context writing and denoising while preserving their interaction through causal attention. Both roles are jointly optimized using the original generation objective without auxiliary losses, with context writing supervised through its contribution to future predictions. This simple change improves visual quality and long-horizon consistency over the evaluated baselines in both framewise and chunkwise generation, without additional video training data or a longer training horizon. Trained on only 5s rollouts, SGF+ supports continuous generation for up to 24 hours without long-video fine-tuning. These results highlight role-specific parameterization as an effective design principle for high-quality autoregressive video generation and native long-horizon extrapolation.
Tsinghua AI