Marco Cuturi

A novel $\mathrm{SemiCat}$ architecture achieves discrete diffusion-level performance on TinyGSM and outperforms one-hot flows at scale by addressing self-conditioning and embedding geometry in continuous diffusion language models.

Distilling long documents into compressed KV caches silently cripples base model capabilities on unrelated prompts, but a lightweight query router or modest off-target data mixing completely eliminates this derailment.

Forget full KV caches: randomly routing attention across layers during training lets you drastically cut memory without hurting performance, and sometimes even helps.

Tri-modal masked diffusion models can now be trained from scratch, achieving strong results in text generation, text-to-image, and text-to-speech, thanks to a systematic exploration of the design space and a novel SDE-based batch size reparameterization.

Factually incorrect SLMs can be made more truthful by teaching them *what* to delegate, not just minimizing loss.