Search papers, labs, and topics across Lattice.
This paper introduces Experience Distillation, a novel approach that allows agents to internalize their interaction histories without requiring additional environment interactions. By leveraging in-context learning, the method retains a significant portion of the learning gains—64.8%—compared to traditional supervised fine-tuning, which only recovers 3.8%. The results demonstrate that Experience Distillation enables agents to achieve performance comparable to classical reinforcement-learning methods while using at least 9.6 times fewer environment samples, highlighting its efficiency in real-world applications.
Retaining over 64% of in-context learning gains without further environment interactions could revolutionize how agents learn from their experiences.
Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents'interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8\% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8\%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least \(9.6\times\) fewer environment samples.
