Search papers, labs, and topics across Lattice.
This paper introduces KEET, an LLM-based agentic framework that interprets NVIDIA Nsight Compute profiles of GPU kernels to provide natural language explanations of performance bottlenecks and optimization suggestions. KEET leverages LLMs to understand complex performance data and generate data-grounded explanations, which are then used to improve LLM-driven code optimization and multiple-choice question answering. Experiments on NVIDIA H100 GPUs show that KEET's explanations enhance LLM performance in downstream tasks and improve the quality of optimization suggestions when analyzing large sets of profiles.
Stop squinting at Nsight Compute profiles: KEET uses LLMs to automatically diagnose GPU kernel bottlenecks and suggest optimizations in plain English.
Performance profiles of GPU kernels generated by tools such as Nsight Compute are rich in detail but are often challenging to interpret. To achieve the best performance possible on a given GPU architecture, kernel developers need to spend significant time analyzing and comparing profiles in the tool's graphical interface to identify and understand kernel performance bottlenecks. Large Language Models (LLMs) have shown promise in understanding complex data and generating natural language explanations. In this paper, we propose the Kernel Execution Explanation Toolkit (KEET), an LLM-based agentic framework for interpreting Nsight Compute profiles to generate useful and data-grounded natural language explanations of performance issues in GPU kernels, and suggestions for optimizations. We evaluate \toolname using several CUDA kernels of varying complexity on NVIDIA H100 GPUs. We find that the generated explanations, when provided as context, improve the quality of LLM code optimization and multiple-choice question answering in downstream tasks. We further demonstrate that the tool can be used to interpret performance data from large sets of profiles to improve the quality of optimization suggestions.
