Search papers, labs, and topics across Lattice.
This study introduces AgentStream, a framework for evaluating self-evolving large language model (LLM) agents in realistic streaming scenarios, addressing the limitations of previous independent evaluations. By systematically organizing agentic benchmarks into configurable task streams and testing five self-evolving methods across three foundation models, the research reveals that the effectiveness of self-evolution is influenced by model capability and varies across different streaming contexts. Key findings indicate that no single self-evolving method consistently outperforms others, emphasizing the need for tailored approaches based on specific model and scenario combinations.
Self-evolving LLM agents show variable reliability in dynamic task environments, challenging the notion that stronger models always yield better adaptation.
Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominantly adopt independent evaluation. Consequently, the behavior of self-evolving agents in realistic streaming settings, where agents adapt to diverse and complex task streams, remains poorly understood. To address this gap, we introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components by organizing agentic benchmarks into a configurable task stream and instantiating the \texttt{Isolated}, \texttt{Sequential}, and \texttt{Interleaved} streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.
