Search papers, labs, and topics across Lattice.
This paper introduces CREDIT, a cost-guided framework that optimizes the use of NVIDIA's distributed shared memory (DSMEM) by identifying profitable workload patterns and predicting their performance benefits. By employing a profiling-driven characterization, a transformation for reduction-reuse workloads, and a cost model based on profiling data, CREDIT consistently outperforms existing compilation methods, achieving significant speedups across diverse workloads. Evaluations demonstrate a 91.7% accuracy in profitability predictions and geometric-mean speedups of 1.466x on RTX 5090 and 1.318x on H100 compared to traditional CUDA baselines.
CREDIT achieves 91.7% accuracy in predicting DSMEM profitability, delivering consistent speedups that outperform leading compilation frameworks.
NVIDIA distributed shared memory (DSMEM) enables direct shared-memory access within a thread block cluster. However, cluster synchronization, remote access, and resource costs make it difficult to determine when DSMEM improves performance. To fill this gap, we propose CREDIT, a cost-guided framework that identifies DSMEM-profitable workload patterns, predicts their profitability range, and delivers consistent speedups across diverse workloads. CREDIT combines three innovations: (1) a profiling-driven characterization that identifies workload patterns likely to benefit from DSMEM; (2) a transformation that applies DSMEM to reduction-reuse workloads; (3) a cost model based on profiling data, to determine its profitability range. Evaluations on diverse workloads show CREDIT achieves 91.7% prediction accuracy on profitability. CREDIT beats torch.compile, Triton, and optimized non-DSMEM CUDA baselines on all six workloads, with geometric-mean speedups of 1.466x on RTX 5090 and 1.318x on H100. CREDIT's source code is publicly available at https://github.com/zhengxiongli08/CREDIT.
