Search papers, labs, and topics across Lattice.
This paper introduces the Chain-of-Experience (CoE) framework, which allows large language models (LLMs) to improve iteratively during test time by leveraging feedback from self-interactions and environmental signals. By evaluating eight LLMs, including GPT-5 and Gemini-2.5 Pro, across various domains, the authors demonstrate that CoE consistently outperforms traditional feedback-free approaches, achieving a 5.6% overall improvement and reducing API costs by 19%. The findings reveal that combining different feedback mechanisms enhances performance, with significant gains observed early in the iterative process, highlighting the potential for continual learning in LLMs.
Iterative experience can boost LLM performance by over 5%, transforming how we evaluate and enhance model capabilities in real-time.
Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models'ability to improve through inference-time interaction. In this paper, we study how LLMs learn from iterative experience at test time, a setting we refer to as Chain-of-Experience (CoE), where models accumulate experiential traces through iterative interactions with self or environmental feedback to form a continual improvement loop beyond zero-shot inference. We instantiate CoE with diverse feedback mechanisms, including model self-feedback and environmental signals such as correctness or public coding test pass rates, and evaluate across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, Claude-4.5 Sonnet. Our study shows that leveraging iterative experience consistently outperforms feedback-free baselines, achieving substantial gains with self feedback alone, alongside a 5.6% overall improvement and 19% lower API cost across tasks and models. We further show that combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and that CoE delivers higher accuracy per token than existing test-time strategies. We observe a positive correlation between LLM base ability and improvement capacity, and show that models remain robust under weak or spurious feedback, with different feedback contributing to distinct improvement aspects and most gains emerging early in the iterations.
UW