Search papers, labs, and topics across Lattice.
This paper addresses the challenges of deploying Mixture-of-Experts (MoE) models on high-bandwidth superpods by introducing UBEP, a communication library designed to optimize the All-to-All communication primitives. The authors identify and tackle three key bottlenecks: execution serialization from Bulk Synchronous Parallel orchestration, synchronization overhead, and load imbalance due to irregular token traffic scheduling. Experimental results demonstrate that UBEP significantly enhances performance, achieving a 52.4% reduction in All-to-All latency and an 11.1% decrease in MoE inference Time Per Output Token (TPOT).
UBEP slashes All-to-All latency by over 52%, unlocking the full potential of Mixture-of-Experts models on superpod architectures.
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.
