Search papers, labs, and topics across Lattice.
This paper investigates the trade-off between domain specialization and general capabilities in large language models using a novel approach called uncertainty-calibrated Multi-Teacher On-Policy Distillation (MOPD). By employing dual-temperature sampling and positive-advantage-density filtering, the authors enhance the selection of training trajectories, leading to more reliable updates that preserve general capabilities during specialization. Experimental results demonstrate that this method improves general capabilities by 4.73% and 10.84% in role-playing and medical-domain tasks, respectively, without sacrificing performance in specialized areas.
Uncertainty-calibrated MOPD boosts general capabilities in specialized language models by over 10% while maintaining domain-specific performance.
Specializing large language models to vertical domains improves domain-specific behavior but often degrades general capabilities such as reasoning, coding, instruction following, and creative writing. We study this domain--general trade-off in Multi-Teacher On-Policy Distillation (MOPD), where a specialized student is supervised on its own sampled trajectories by domain and general teachers. Standard MOPD faces two limitations: ordinary on-policy sampling rarely exposes tokens with large positive teacher--student advantages, while the advantage sign alone does not establish whether the resulting update direction is reliable. We propose uncertainty-calibrated MOPD to address these limitations. Dual-temperature sampling broadens the candidate trajectory pool, and positive-advantage-density filtering selects trajectories with stronger positive learning signals. Centered log-likelihood (CLL) filtering then computes an entropy-calibrated teacher-endorsement score and probabilistically retains token updates according to direction--endorsement consistency. Experiments on role-playing and medical-domain specialization show that our method improves the general-capability average over standard MOPD by $4.73\%$ and $10.84\%$, respectively, while maintaining vertical-domain performance. Ablations and diagnostic analyses further confirm that the gains do not merely result from a larger rollout budget and that the proposed trajectory- and token-level mechanisms address their intended failure modes.
