Search papers, labs, and topics across Lattice.
This paper introduces ASPIRE, a benchmark designed to facilitate self-evolution in models driven by vague goals rather than explicit tasks. By allowing agents to interpret and operationalize ambiguous objectives, the study reveals that current models struggle with weight-level improvements and often fail to generalize their learning to hidden evaluations. The findings indicate that while agents can effectively navigate training and harness-editing loops, their reliance on narrow self-evaluations leads to unstable progress and a lack of transferability in skill development.
Vague goals can misdirect model evolution, revealing that agents often overfit to narrow self-assessments, hindering broader learning outcomes.
Many important forms of human learning begin with a vague goal, such as"become a better physicist"or"improve at research."Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution. ASPIRE provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation. Current agents routinely complete training and harness-editing loops, but weight-level gains remain sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference. Agents often train on mismatched data and trust narrow self-evaluations, so local gains fail to transfer to hidden evaluation and continued search and training can erase earlier improvements.
