John Thickstun

Auxiliary draft models are no longer necessary for speculative decoding: distilling lightweight diffusion heads directly into standard LLMs yields lossless 3× generation speedups even at peak batch sizes.

Low diversity in training data can lead to substantial performance drops in language models, revealing a critical oversight in data augmentation practices.

Forget runtime inference costs: "compiled AI" slashes token consumption by 57x while maintaining accuracy by pre-generating deterministic code from LLMs.