Search papers, labs, and topics across Lattice.
This paper introduces PCoMoE, a novel framework that transitions Mixture-of-Experts (MoE) inference from a rigid, monolithic expert selection approach to a more flexible fine-grained path composition method. By implementing a path-level formulation of expert computation alongside a compatibility-aware layer-wise pruning strategy, PCoMoE effectively reduces computational redundancy and enhances efficiency. Experimental results show that this approach not only accelerates end-to-end inference speed by up to 1.31x but also improves model accuracy by 10%, indicating significant advancements in MoE architecture optimization.
Shifting from monolithic expert selection to fine-grained path composition, PCoMoE boosts inference speed by 1.31x while enhancing accuracy by 10%.
Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. However, modern MoE inference remains heavily constrained by the rigid, whole-expert abstraction. Existing frameworks manage, schedule, or prune experts as atomic execution units, which fixes the optimization boundary too early and leaves fine-grained intra-expert computational redundancy underexplored. In this work, we present PCoMoE, a path-compositional execution framework that shifts MoE inference from coarse-grained expert selection to fine-grained path composition. PCoMoE incorporates a path-level formulation of expert computation, a compatibility-aware layer-wise pruning strategy to suppress low-value path combinations, and a hardware-friendly execution engine to exploit reusable sub-expert structures under strictly bounded overheads. Experimental results demonstrate that PCoMoE achieves up to a 1.31x end-to-end inference speedup while enhancing model accuracy by 10%. The code is available at https://github.com/gzyyy0/PCoMoE
