Search papers, labs, and topics across Lattice.
DAMOGuangdong Provincial Key Laboratory of VisualHUSTShenzhen UniversityJun 25, 2026arXiv:2606.26938This paper addresses a critical routing assignment issue in existing Mixture-of-Experts (MoE) architectures for diffusion models, where the router inadequately allocates resources to salient tokens due to reliance on noise-corrupted latent features. The authors introduce SharpMoE, a post-training framework that leverages clean latent features for accurate routing, effectively guiding the router to identify salient tokens even in high-noise conditions. Experimental results show that SharpMoE significantly enhances the performance of pretrained MoE models, achieving state-of-the-art results in visual generation tasks.
By harnessing clean latent features, SharpMoE dramatically improves resource allocation to salient tokens, overcoming a key limitation in diffusion MoE architectures.
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.
