Search papers, labs, and topics across Lattice.
This study investigates the effects of privileged self-distillation on thinking models, revealing that while self-distillation is generally beneficial, it can lead to significant performance degradation on long reasoning tasks, with accuracy dropping by up to 17% in certain scenarios. The authors find that withholding privileged context negatively impacts the model's ability to explore multiple plausible reasoning paths, particularly in high-entropy situations. The research highlights a critical distinction in how privileged context influences learning in thinking versus instruction-tuned models, suggesting that self-distillation strategies need to be carefully tailored for optimal performance in complex reasoning tasks.
Privileged self-distillation can paradoxically hinder thinking models, leading to a 17% drop in accuracy on long reasoning tasks due to its impact on learning dynamics.
Self-distillation is a promising recipe for self-improvement in language models. In this setting, a model can serve as its own teacher when given privileged information, such as a solution to a math problem. This seems especially appealing for thinking models, which can use test-time reasoning to absorb the privileged information. Surprisingly, we show that privileged self-distillation degrades thinking models on long reasoning traces: across five Qwen3 and OLMo thinking models evaluated on AIME24, AIME25, and HMMT25, privileged-context distillation causes a relative drop of up to 17% in avg@16 accuracy. The degradation scales with the amount of privileged context withheld from the student and is most pronounced at long rollout budgets, where thinking models otherwise obtain their largest gains. This failure mode is not specific to self-distillation: on-policy distillation (OPD) improves thinking models, but privileged OPD reverses these gains. Our diagnostics link this failure mode to how privileged teacher context reshapes learning at high-entropy forking positions, where multiple continuations remain plausible and may lead to different reasoning paths. Privileged context lowers fork rates in thinking-model rollouts but not in instruction-model rollouts. This leads to an interesting dichotomy, where privileged context can help instruction-tuned models but hurts stronger thinking models. The effect is visible when the student begins a self-correction branch, where privileged OPD penalizes sampled reconsideration tokens that vanilla OPD supports. Thinking models trained with a privileged teacher produce fewer verification, backtracking, and hedging markers, even after length normalization. These findings indicate that self-distillation for strong thinking models requires attention to token-level signal, especially around correction and reasoning steps.
