Search papers, labs, and topics across Lattice.
This paper introduces NeutronSparse, a novel framework designed to enhance sparse matrix-matrix multiplication (SpMM) performance on Neural Processing Units (NPUs) by addressing coordination challenges among heterogeneous compute units. By implementing sparsity-aware workload partitioning and locality-aware tile orchestration, NeutronSparse significantly improves data management and execution efficiency. Evaluation results demonstrate that NeutronSparse achieves speedups of 1.26x-7.78x over existing NPU baselines and 1.03x-3.07x over top GPU libraries, highlighting the potential of NPUs in sparse data processing.
NeutronSparse reveals that with the right coordination strategies, NPUs can outperform leading GPU libraries in sparse matrix operations.
Sparse matrix-matrix multiplication (SpMM) is a fundamental data operation for large-scale sparse data processing. With NPUs increasingly deployed in data centers for their performance and energy efficiency, accelerating SpMM on these platforms is a natural choice. However, high-performance SpMM on NPUs poses a data management challenge, as irregular sparsity demands efficient data organization and scheduling. On Ascend 910B, the official MindSpore implementation achieves only 36.3% of the performance of GPU-based sparse libraries such as cuSPARSE on NVIDIA A100. To this end, we conduct an in-depth architectural analysis of SpMM execution on NPUs versus GPU and identify that the key performance bottleneck for SpMM on NPUs lies in the lack of efficient coordination across heterogeneous compute units under tile-based execution model. Therefore, we propose NeutronSparse, a coordination-first SpMM framework for NPUs. NeutronSparse integrates two key techniques: (i) Sparsity-aware coordination of heterogeneous engines, which adaptively partitions and balances workloads between heterogeneous compute units to keep them busy, and (ii) Locality-aware tile orchestrating, which reorganizes and reuses data tiles to reduce redundant computation and memory movement overhead. Evaluations on Ascend 910B show that NeutronSparse achieves 1.26x-7.78x speedup over NPU baselines and 1.03x-3.07x speedup over leading GPU libraries on NVIDIA A100, revealing untapped potential of NPUs for sparse computation.
