Search papers, labs, and topics across Lattice.
The paper introduces SEAL, a closed-loop co-evolution framework that synergistically adapts both the LLM agent policy and its learning environment for interactive tool use. SEAL diagnoses failed rollouts into turn-level failure labels, using these as a shared signal to evolve the environment's training interface (e.g., clearer tool affordances) and update the policy with diagnosis-guided advantage reweighting. Experiments show that SEAL significantly improves low-resource agent learning, achieving substantial gains in both in-distribution and out-of-distribution multi-turn tool-use evaluations with only 400 training samples.
LLM agents learn tool use far more efficiently when their learning environment evolves alongside them, adapting to the agent's changing capabilities.
Large Language Model (LLM) agents are increasingly improved through interaction, yet most self-evolution methods adapt either the policy or the learning environment in isolation. We identify this structural gap as Agent-Environment Misalignment: the agent's capability frontier changes during training, while the environment that provides supervision remains static or only weakly coupled to the agent's revealed failures. We propose SEAL, a closed-loop co-evolution framework for interactive tool-use agents. SEAL collects on-policy trajectories under executable verification, diagnoses failed rollouts into turn-level failure labels, and uses these diagnoses as a shared signal for both environment-side adaptation and model-side policy optimization. The environment evolves its training-time learning interface by exposing clearer tool affordance cues, constraint information, and recovery-oriented feedback, while the policy is updated with diagnosis-guided advantage reweighting. Extensive experiments across in-distribution and out-of-distribution multi-turn tool-use evaluations show that SEAL improves low-resource agent learning: with only 400 training samples, it yields +8.25 to +26.25 average-point gains across three backbones and exhibits positive out-of-distribution transfer. These results demonstrate the value of jointly adapting the learner and its training-time learning substrate for robust self-improving LLM agents.
