Search papers, labs, and topics across Lattice.
This paper introduces SparseDitto, an LLM-based system that customizes GPU kernels for sparse matrix operations by adapting to various sparsity patterns and execution strategies. By leveraging a lightweight additive model to rank strategies and an architecture-aware planner to propose designs, SparseDitto achieves significant performance improvements, with a geometric mean speedup of 2.68x over cuSPARSE on an NVIDIA RTX PRO 6000 GPU. The system not only enhances the efficiency of sparse matrix operations like SpMV, SpMM, and SpGEMM but also accelerates full-batch GCN training by up to 3.39x, highlighting its practical impact on deep learning workloads.
SparseDitto achieves up to 146.61x speedup for sparse matrix operations by dynamically customizing GPU kernels based on input patterns.
Sparse matrix kernels are fundamental to scientific computing, graph analytics, and machine learning. Their GPU performance depends strongly on the input sparsity pattern and execution strategy. For the same SpMM on the same matrix, cuSPARSE exhibits a 350x performance gap between CSR and Blocked-ELL. Our study of multiple data formats, specialized systems, and sparse compilers shows that no single implementation consistently dominates across sparsity patterns and operators. This motivates a system that can adapt its representation, execution strategy, and hardware mapping to each workload and target GPU. We present SparseDitto, an LLM-based system that constructs a GPU kernel for each matrix, operator, and target GPU. SparseDitto supports SpMV, SpMM, and SpGEMM within a unified design framework. A lightweight additive model ranks established strategies using structural features of the input matrix. An architecture-aware planner then proposes several candidate designs. Coding and verification agents implement and refine them using measurements from the target GPU. Across three sparse operators and a diverse set of matrices, SparseDitto achieves a geometric-mean speedup of 2.68x over cuSPARSE on an NVIDIA RTX PRO 6000 GPU, with a maximum of 146.61x. On an NVIDIA H200 GPU, it achieves 2.79x, with a maximum of 78.5x. Its generated SpMM kernels also accelerate full-batch GCN training by up to 3.39x.
