Search papers, labs, and topics across Lattice.
The paper introduces Anchored Learning, a novel supervised fine-tuning (SFT) framework that mitigates catastrophic forgetting in LLMs by explicitly controlling distributional drift during optimization. Anchored Learning uses a dynamically evolving moving anchor that interpolates between the current model and a frozen reference, effectively transforming global fine-tuning into a sequence of local trust-region updates. Empirical results on iGSM, MedCalc, and IFEval demonstrate that Anchored Learning achieves a superior gain-stability trade-off, significantly reducing performance degradation compared to standard SFT while maintaining near-optimal performance improvements.
LLMs can retain 10x more of their original capabilities after fine-tuning, simply by using a dynamically adjusted "anchor" to constrain distributional drift during training.
Post-training large language models (LLMs) often suffers from catastrophic forgetting, where improvements on a target objective degrade previously acquired capabilities. Recent evidence suggests that this phenomenon is primarily driven by excessive distributional drift during optimization. Motivated by this perspective, we propose Anchored Learning, a simple framework that explicitly controls distributional updates during offline fine-tuning via a dynamically evolving moving anchor. Instead of matching a fixed reference distribution, the anchor interpolates between the current model and a frozen reference to construct an intermediate target that the model distills toward, transforming global fine-tuning into a sequence of local trust-region updates in distribution space. Theoretically, we prove this anchor-based update admits a linear KL-divergence upper bound per iteration, ensuring a stable transition between model distributions. Extensive experiments on iGSM, MedCalc, and IFEval show that Anchored Learning consistently lies on the Pareto frontier of gain-stability trade-offs, achieving near-optimal performance improvements while substantially reducing degradation compared to strong baselines. For example, while standard SFT suffers from over 53% performance degradation on iGSM and MedCalc, Anchored Learning slashes this drop to under 5% while maintaining near-optimal gains (e.g., 75.2% on iGSM).
