Frederic Sala

This work introduces a unified multi-agent framework, MAGS, that generates executable programs with formal safety guarantees, using Dafny as a verification-aware intermediate representation where safety properties can be mechanically checked.

Registers are implemented as dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks, which are especially effective for bounded code generation, where correct programs usually span several chunks.

Current LLMs often overfit and fail to leverage past knowledge, with naive in-context learning outperforming dedicated memory systems in continual learning tasks.

Agents can automate the data curation loop but struggle to innovate without structured guidance, revealing a critical gap in their capabilities.

Optimal LLM pretraining actually requires *overtraining* when you account for inference costs, overturning conventional scaling wisdom.

SkillOrchestra slashes the learning costs of AI agent orchestration by up to 700x while improving performance by explicitly modeling agent skills and costs, offering a more scalable and interpretable alternative to RL-based methods.