Search papers, labs, and topics across Lattice.
This paper introduces StrataCL, a fabric-native communication library designed to optimize communication in distributed AI workloads by eliminating redundant data copies through user-buffer direct communication. By leveraging registration-on-allocation and optimizing communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading, StrataCL significantly enhances performance on the Huawei CloudMatrix384. The results demonstrate substantial improvements in collective bus bandwidth and LLM inference throughput, achieving up to 1.9x throughput gains and reducing training iteration times across various workloads.
StrataCL achieves up to 1.9x faster LLM inference and significantly reduces training times by streamlining communication in distributed AI systems.
Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because user and communication buffers are managed separately, causing redundant data copies or costly user-buffer registration. This paper presents StrataCL, a zero-redundancy and fabric-native communication library for production supernodes. StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features. On the Huawei CloudMatrix384, StrataCL improves collective bus bandwidth by up to 1.6x and improves MoE dispatch/combine bus bandwidth by up to 1.4x. Across three production workloads, StrataCL improves LLM inference throughput by 1.9x, reduces P99 TTFT by 2.2x, and reduces LLM and Recsys training iteration time by 1.4x and 1.3x, respectively.
