Search papers, labs, and topics across Lattice.
This paper introduces EGG, an Expert-Guided Agent Framework designed to enhance the generation of high-performance GPU kernels by integrating expert optimization principles into the decision-making process of large language models (LLMs). By decomposing the kernel generation task into two hierarchical stages鈥攁lgorithmic structure design and hardware-specific tuning鈥擡GG establishes clear optimization objectives that facilitate effective exploration of the optimization space. Experimental results demonstrate that EGG achieves a 2.13x average speedup over PyTorch, significantly outperforming existing agent-based and reinforcement learning approaches in kernel generation tasks.
EGG achieves a remarkable 2.13x speedup in GPU kernel generation, setting a new benchmark for performance in automated optimization.
High-performance GPU kernels are critical for reducing the exponentially growing computational costs of large language models (LLMs), but their development heavily relies on manual tuning by domain experts. While recent advances in LLM-based approaches show promise for automating kernel generation, they still struggle to achieve both correctness and high performance. This limitation primarily arises from the lack of domain-specific optimization guidance, hindering effective exploration of the optimization space. We propose EGG, an Expert-Guided Agent Framework for Kernel Generation, which incorporates expert optimization principles to guide LLMs' decisions. Inspired by expert workflows, we decompose kernel generation into two hierarchical stages: 1) algorithmic structure design, which establishes a high-quality computational structure foundation; 2) hardware-specific tuning, which performs targeted adjustments through parallel mapping, tensor tiling, and memory optimization. This staged decomposition defines explicit optimization objectives, structuring the design space to achieve progressive refinement. To this end, a stage-aware multi-agent collaboration mechanism is designed for inter and intra-stage context management, ensuring stable optimization trajectories. Experiments on KernelBench and real-world workloads show that EGG achieves a 2.13x average speedup over PyTorch, outperforming existing agent-based and RL-based approaches.
Tsinghua AI