Search papers, labs, and topics across Lattice.
This paper identifies three key failure modes in LLM knowledge distillation: tail noise, off-policy instability, and teacher-student gap, leading to issues like overconfident hallucinations in student models. To address these, they introduce a reinforcement fine-tuning (RFT) based calibration method that allows explicit control over a teacher model's distillability. Experiments demonstrate that this approach improves student performance when distilling from "distillable" teachers and effectively prevents knowledge transfer from "undistillable" teachers, offering a tunable safety mechanism.
You can now dial a knob to make your LLM either super-distillable or completely un-distillable, opening up new possibilities for both efficient knowledge transfer and robust model protection.
Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks. Our analysis revealed several distillation traps: tail noise, off-policy instability, and, most fundamentally, the teacher-student gap, that distort training signals. These traps manifest as overconfident hallucinations, self-correction collapse, and local decoding degradation, causing distillation to fail. Motivated by these findings, we propose a post-hoc calibration method that, to the best of our knowledge, for the first time enables control over a teacher's distillability via reinforcement fine-tuning (RFT). Our objective combines task utility, KL anchor, and across-tokenizer calibration reward. This makes distillability a practical safety lever for foundation models, connecting robust teacher-student transfer with deployment-aware model protection. Experiments across math, knowledge QA, and instruction-following tasks show that students distilled from distillable calibrated teachers outperform SFT and KD baselines, while undistillable calibrated teachers retain their task performance but cause distilled students to collapse, offering a practical knob for both better KD and model IP protection.
