Search papers, labs, and topics across Lattice.
This paper introduces Contrastive Reflection (CORE), a non-parametric learning algorithm that enhances reasoning in language models by generating insights from past reasoning traces. CORE significantly reduces the need for extensive training samples and rollouts, achieving rapid improvements across four reasoning tasks while maintaining or exceeding the performance of existing methods. The findings indicate that distilling contrasts between successful and unsuccessful attempts into natural-language insights offers a more efficient and interpretable approach to model self-improvement than traditional techniques.
CORE enables language models to rapidly enhance reasoning capabilities using minimal training data and rollouts, outperforming traditional methods.
Language models can use verifiable rewards to improve at a wide variety of reasoning tasks. However, both parametric (e.g. RLVR) and non-parametric (e.g. prompt optimization) approaches to doing so typically require hundreds of training samples and thousands of model rollouts, making them expensive in the best case and intractable in the worst. To address this challenge, we introduce Contrastive Reflection (CORE), a non-parametric learning algorithm that compares past reasoning traces to generate insights: short natural-language descriptions of reasoning strategies and constraints that capture differences between successful and unsuccessful problem attempts. Across four reasoning tasks, we demonstrate that CORE enables more rapid improvement than both parametric (GRPO) and non-parametric (GEPA, episodic RAG, and MemRL) methods, while using fewer rollouts. Under fixed rollout budgets with as few as five training samples, we then show that CORE also achieves comparable or greater performance gains than each baseline. Finally, we highlight how CORE is also substantially more context-efficient than non-parametric baselines, requiring fewer prompt tokens while storing learned knowledge as compact, interpretable natural-language insights. Our results therefore suggest that distilling contrasts between successful and unsuccessful reasoning traces into abstract and useful insights can provide a more efficient and interpretable route to model self-improvement than weight updates, prompt optimization, or direct reuse of stored reasoning traces.
Stanford HAI