Search papers, labs, and topics across Lattice.
This paper introduces EvoAgentBench, a novel benchmark designed to evaluate agent self-evolution through Ability-guided transfer across four domains: web research, algorithmic reasoning, software engineering, and knowledge work. The benchmark focuses on the procedural reuse of experiences rather than mere information retention, enabling a more nuanced assessment of how agents leverage learned abilities. Results indicate that while curated Ability content can be transferred reliably across different model families, no existing automatic method consistently achieves positive gains across all evaluation settings, highlighting the complexity of self-evolution in agents.
Ability-guided transfer reveals that agents struggle to consistently leverage learned experiences, challenging the effectiveness of current self-evolution methods.
Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.
