Search papers, labs, and topics across Lattice.
This study investigates the performance gap between correctness and efficiency in GPU domain-specific languages (DSLs) by analyzing 22 kernels from Triton and TileLang on NVIDIA GPUs. It reveals that correctness-based evaluations can overlook significant performance issues, with some kernels running over 300 times slower than their optimized counterparts despite passing correctness checks. The authors propose two lightweight screening methods, library-relative efficiency and roofline utilization, which effectively identify inefficient kernels and distinguish between repairable defects and inherent structural limitations.
Correctness checks can miss kernels that are functionally valid but over 300 times slower than optimized versions, highlighting a critical evaluation gap in GPU DSLs.
Modern GPU domain-specific languages (DSLs), such as Triton and TileLang, are increasingly used to implement specialized deep-learning kernels and as target languages for automated kernel-generation systems. Existing DSL-kernel evaluations establish correctness through reference-based numerical validation -- necessary, but silent on replacement quality: a functionally valid kernel may still fall far below the throughput of the optimized library operator it is intended to replace. We study this correctness-performance gap using 22 Triton and TileLang kernels from five operator categories on NVIDIA A100 and GH200 GPUs, asking whether correctness-based evaluation identifies kernels unsuitable as library replacements, why such failures occur, and how they can be detected without exhaustive benchmark coverage. The study yields three results. \emph{First}, correctness-based evaluation can admit severe slowdowns: an idiomatic TileLang LayerNorm kernel passes KernelBench's correctness check while running more than 300$\times$ slower than the PyTorch baseline. \emph{Second}, the causes differ by kernel family. TileLang normalization and reduction slowdowns are mainly repairable authoring defects, such as sequential reductions and unnecessary dtype conversions, whereas convolution and large general matrix multiplication (GEMM) retain residual gaps after optimization due to code-generation and autotuning-coverage limits; vendor-library algorithm selection contributes only marginally. \emph{Third}, two lightweight checks -- library-relative efficiency and roofline utilization -- are complementary screening criteria: together they flag every functionally valid but inefficient kernel in our suite and separate repairable authoring defects from structural residuals.
