Search papers, labs, and topics across Lattice.
The paper introduces Tree-of-Experience (ToE), a hierarchical experience-management framework designed to enhance the self-evolution of LLM agents by aligning experience organization with their reasoning processes. By structuring experiences into a tree of analytical perspectives and reasoning paths, ToE allows for more effective feedback attribution, cross-task transfer, and retrieval efficiency, particularly in complex reasoning tasks. Experimental results demonstrate that ToE significantly outperforms traditional experience-management methods, achieving a 31.4% relative accuracy improvement on the Game of 24 and a 41.24% improvement in task-specific incremental change on FinEvolveBench.
Transforming how LLM agents manage experiences, ToE achieves remarkable gains in problem-solving efficiency and accuracy, outpacing conventional methods.
Continual self-evolution requires LLM agents to transform environmental interactions into reliable and reusable experience. Existing methods typically refine individual trajectories or abstract shared knowledge from related trajectories, but their experience representations are often disconnected from the underlying reasoning process. This limits feedback attribution, cross-task transfer, and update and retrieval efficiency, particularly in complex reasoning tasks with outcome-level feedback. To overcome this limitation, we propose \textbf{T}ree-\textbf{o}f-\textbf{E}xperience (ToE), a structured experience-management framework that aligns experience organization with the hierarchical reasoning process of LLM agents. Specifically, ToE organizes the experience into a shared tree of analytical perspectives and reasoning paths, whose reliability is calibrated through environmental outcomes to support systematic updating, transfer, and efficient retrieval. The experimental results on \textsc{Game of 24} and \textsc{FinEvolveBench} show that ToE substantially improves both problem-solving performance and efficiency. On \textsc{Game of 24}, ToE achieves a 31.4\% relative improvement in accuracy over the experience-free ToT baseline. On \textsc{FinEvolveBench}, ToE improves tsIC by an average of 41.24\% over the experience-free pipeline across 12 evaluation settings, whereas conventional experience-management methods often underperform experience-free baselines.
