Search papers, labs, and topics across Lattice.
This paper introduces Agents-A1, a 35B Mixture-of-Experts model that achieves performance comparable to trillion-parameter models by focusing on scaling the agent horizon rather than increasing parameter count. The authors develop a long-horizon knowledge-action infrastructure that enables the model to generate agentic trajectories averaging 45K tokens in length, and employ a three-stage training process that includes supervised fine-tuning, domain-level teacher models, and a novel multi-teacher distillation approach. Agents-A1 outperforms existing models on several long-horizon benchmarks, demonstrating that significant performance gains can be achieved without the need for larger parameter sizes.
Achieving trillion-parameter performance with just 35 billion parameters by scaling agent horizons reveals a new frontier in model efficiency.
We introduce Agents-A1, a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities. To support this goal, we build a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45K tokens. Based on this, we train Agents-A1 with a three-stage recipe. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to improve knowledge transfer efficiency across different domains, unifying six heterogeneous domains into one deployable student model. Agents-A1 achieves strong and broad performance for long-horizon agent benchmarks. Compared with 1T-parameter model such as Kimi-K2.6 and DeepSeek-V4-pro, Agents-A1 achieves leading results on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and remains highly competitive on SciCode (44.3), HLE (47.6) and BrowseComp (75.5). We hope this work provides the community with a practical path for scaling the horizon using a 35B agent that can reach or match the performance of 1T models on long-horizon tasks.
