Luke Zettlemoyer

Language models that edit their own context window like a mutable file beat external agent scaffolding across long-horizon tasks while cutting compute by up to 59%.

Image models do not need noisy vision-language reward models to master compositional reasoning: training on programmatically verified synthetic scenes yields a 10x boost in spatial accuracy that transfers directly to natural prompts.

This work defines the scaling inflection point as the per-session budget where marginal Elo gains match the independent-sampling reference, and proposes Elo-per-token analysis, which tracks the best solution found at each token budget and uses a Bradley-Terry model to aggregate within-task orderings into Elo ratings across tasks with different score scales.

This work addresses the adverse interactions between sparsity and data repetition, and presents evidence for the core mechanisms of overfitting and its potential remediation, and suggests promising avenues for future methods to reduce over-specialization in model parameters by disrupting memorization patterns.

Switch Distillation not only enhances reasoning performance by up to 71% but also preserves factual recall, challenging the conventional trade-off in knowledge distillation.

By making environment design a learnable process, SPADE unlocks a new frontier in self-improvement for language agents, leading to substantial performance gains across diverse tasks.

M* achieves up to 2.9x lower real-time factor and 2.7x higher throughput for text-to-speech tasks, revolutionizing how we serve complex multimodal AI models.

Instant AI assistants are now feasible on smartwatches: 8M-parameter models can kickstart responses locally, hiding cloud latency with surprisingly high quality.

Learning robotic reward functions from a million trajectories reveals that comparing entire trajectories, not just individual frames, unlocks better generalization and learning from suboptimal data.