Subham Sekhar Sahoo

Auxiliary draft models are no longer necessary for speculative decoding: distilling lightweight diffusion heads directly into standard LLMs yields lossless 3× generation speedups even at peak batch sizes.

Uniform-state diffusion models can now achieve state-of-the-art generative performance in language modeling, thanks to a new Predictor-Corrector sampler that breaks the quality plateau of ancestral sampling.

Uniform-state diffusion models, often overlooked in favor of masked diffusion, surprisingly outperform autoregressive and masked diffusion models on GSM8K when scaled to 1.7B parameters, despite worse perplexity.