Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces MoEvolve, a bidirectional distillation framework that leverages the inherent latent capacity of Mixture-of-Experts (MoE) models by increasing their activation budget to create a stronger internal teacher. The method first recovers performance in the expanded model and then distills its enhanced capabilities back to the original low-$k$ configuration, resulting in significant performance gains. MoEvolve improves initial MoE models by up to $22.2$ domain-average points and outperforms same-budget RLVR baselines by up to $6.5$ points.
Activating latent expert capacity within MoE models via bidirectional distillation unlocks performance gains exceeding even the expanded teacher, pushing the boundaries of efficient scaling.
Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved Mixture-of-Experts (MoE) LLMs, but its outcome-level rewards provide only sparse supervision. On-Policy Distillation offers dense token-level feedback, yet requires a stronger compatible teacher, while on-policy self-distillation relies on hand-designed privileged information. We observe that the MoE architecture itself offers a way to resolve this dilemma, as increasing the activation budget per token exposes latent expert capacity and yields a potentially stronger teacher within the model. However, naive expansion degrades performance because newly co-activated experts are not optimized to work together. We therefore propose MoEvolve, a bidirectional distillation framework following a recovery-then-improvement strategy: we first restore the activation-expanded high-$k$ model and further strengthen it by exploiting its additional active capacity, then reverse the direction of distillation to transfer the enhanced capabilities back into the original low-$k$ configuration, retaining its inference-time activation budget. Across diverse MoE models and benchmarks, MoEvolve improves the initial models by up to $22.2$ domain-average points and outperforms same-budget RLVR baselines by up to $6.5$ points. Notably, the final low-$k$ models often surpass their activation-expanded teachers.