Search papers, labs, and topics across Lattice.
This paper introduces TRUSTEE, a novel method for training tool-calling agents using dynamic environments simulated by free open-source language models (LMs) as small as 8B, eliminating the need for annotated data or complex commercial models. By employing an adaptive curriculum learning mechanism, TRUSTEE dynamically adjusts task difficulty during training, leading to consistent performance improvements across various domains. The results demonstrate that even a relatively simple LM can establish a strong baseline for tool learning, challenging the reliance on more resource-intensive approaches.
Training tool-calling agents with just an 8B language model outperforms traditional methods that depend on expensive resources, reshaping the landscape of tool learning.
Reinforcement learning (RL) has become a prevalent paradigm for training tool calling agents, which typically requires online interactive environments. Existing approaches either rely on training data with ground truth annotations or require advanced commercial language models (LMs) to synthesize environments that keep fixed once created. In this work, we propose TRUSTEE, a data-free method training tool calling agents with dynamic environments fully simulated by free open-source LMs that can be as small as 8B, including task generation, user simulation, tool simulation and trajectory evaluation, paired with an adaptive curriculum learning mechanism that controls various aspects of the task difficulty dynamically during training. Our empirical results show that TRUSTEE brings consistent improvements across various domains and outperforms all the baselines which require extra external resources for training. These confirm that, with a sufficiently sophisticated design, even simulated environments with a local 8B LM as the backbone could set a strong baseline for tool learning, without expensive annotated data, realistic human interactions, executable tools or costly verifiable environments from human experts or commercial LMs. We hope our proposed paradigm could inspire future research on environment scaling with limited resources.
