Song Han

Autoregressive video pretraining unlocks the full potential of long contexts for real-time robot control, boosting success rates by 15% and enabling complex manipulation previously impossible.

Language, when self-evolving and treated as a physical representation, can unlock state-of-the-art physical plausibility in video world models, even surpassing proprietary systems.

Sub-quadratic linear attention no longer requires full-precision state caching: 8-bit recurrent states can match FP32 accuracy while delivering up to 3.70× kernel speedups without any retraining.

High-resolution 4K video upscaling no longer requires iterative diffusion: a single denoising step can outperform multi-step refiners while slashing latency by nearly 9×.

Native 8-bit LLM training silently destabilizes at billion-parameter scale due to softmax gradient drift, but restoring the zero-row-sum invariant completely eliminates the FP8 attention gap without architecture modifications.

Massive 33B video diffusion models can run 3.5x faster than real time by replacing VAE decode-reencode cycles with direct latent-to-latent resolution transfers and automated kernel synthesis.

Distilling causal LLMs into parallel diffusion models fundamentally handicaps students when the teacher ignores visible future tokens—aligning this context yields up to 4-point accuracy gains at 1.5× faster training.