Search papers, labs, and topics across Lattice.
This paper introduces DualOPSD, an innovative framework for on-policy self-distillation that adapts both the student and privileged teacher models during training. By allowing the teacher to adjust to the evolving student distribution without requiring additional rollouts, DualOPSD significantly enhances the learning process. Experiments on the Qwen3-8B model demonstrate that this approach yields substantial improvements in accuracy across various benchmarks, with gains of up to 23.61 points compared to traditional OPSD methods.
Adaptive self-distillation can boost model performance by over 23 points without extra rollouts, reshaping how we approach teacher-student dynamics in training.
On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this privileged teacher fixed, even though the student distribution and output style change during training. We propose DualOPSD, an asymmetric alternating framework that adapts both policies. The student first learns from the privileged teacher. The teacher then moves toward the updated student distribution on the same student trajectory. This update makes later supervision responsive to the learner and does not require another rollout. On Qwen3-8B in non-thinking mode, DualOPSD improves avg@12 over OPSD by 23.61, 13.89, and 10.00 points on AIME 2024, AIME 2025, and HMMT 2025. Results at 1.7B and 4B show that the accuracy gain depends on model scale. Across all three scales, DualOPSD reduces truncation. The 4B diagnostic also shows lower KL in both directions between the teacher and student.
