Search papers, labs, and topics across Lattice.
This paper addresses the problem of recovering general capabilities in domain-specialized LLMs using Multi-Teacher On-Policy Distillation (MOPD) when the general teacher's training data is unknown. They identify recovery-preservation counteraction and weak-signal flattening as key failure modes of vanilla MOPD under incomplete prompt coverage. To mitigate these issues, they propose Counteraction-Aware MOPD (CaMOPD), which uses decoupled alternating training and gap-based sample selection to improve gradient coherence and concentrate correction signals.
Domain-specialized LLMs can regain lost general skills without sacrificing their expertise, thanks to a new distillation method that disentangles conflicting training signals.
Domain specialization can improve LLM behavior in vertical domains, but often weakens the general capabilities inherited from the original model. Recent Multi-Teacher On-Policy Distillation (MOPD) pipelines recover model capabilities by supervising student-generated trajectories with teacher feedback, but typically assume teacher-aligned prompt coverage, requiring prompts to match the teachers' training distributions. This assumption is difficult to satisfy when the general teacher is an open-source model whose post-training data are unknown. Instead of attempting to reconstruct this hidden distribution, we study general capability recovery with readily available proxy general prompts. We identify two failure modes of vanilla MOPD in this incomplete-coverage situation: recovery-preservation counteraction from mixing conflicting recovery and preservation gradients, and weak-signal flattening from uniformly averaging samples with unequal correction demand. We propose Counteraction-Aware Multi-Teacher On-Policy Distillation (CaMOPD), which addresses these issues with decoupled alternating training and gap-based sample selection. CaMOPD gives general recovery dedicated updates, periodically reviews domain prompts for preservation, and selects samples with larger averaged token-level teacher-student log-probability gaps to concentrate correction signals. Across role-play dialogue and medical reasoning QA scenarios, CaMOPD performs best in general recovery over baselines while maintaining domain-specific behavior. Gradient coherence analyses further support the intended effect of CaMOPD in producing more coherent correction signals.
