Search papers, labs, and topics across Lattice.
This paper introduces ComFuse, an automated GPU compilation system that optimizes the execution of complex computation graphs by fusing compute-intensive operators with memory-intensive subgraphs. By allowing concurrent execution of memory-intensive operations alongside compute-intensive ones, ComFuse addresses the limitations of existing compilers that optimize these operator classes separately, thus enhancing on-chip data reuse and performance. Experimental results demonstrate that ComFuse-generated kernels significantly outperform those produced by TorchInductor in various complex computation scenarios, showcasing its ability to support flexible fusion patterns and reduce manual kernel engineering efforts.
ComFuse achieves a breakthrough in GPU compilation by enabling concurrent execution of memory-intensive and compute-intensive operations, leading to significantly improved performance in complex workloads.
Modern deep learning workloads increasingly comprise heterogeneous computation graphs that combine compute-intensive operators with memory-intensive subgraphs. Existing deep learning compilers typically optimize these operator classes separately, creating rigid fusion boundaries that limit cross-operator optimization and on-chip data reuse. We observe that downstream memory-intensive operations can execute concurrently with compute-intensive operators, allowing their execution to be hidden behind computation; however, automatically exploiting this opportunity poses new compilation challenges. In this paper, we present ComFuse, an automated GPU compilation system that employs a novel operator fusion strategy to generate high-performance kernels for complex graph structures comprising compute-intensive operators and dependency-rich, memory-intensive elementwise-reduction subgraphs. ComFuse further supports the fusion of back-to-back GEMM (B2BGEMM) patterns, extending its applicability to more complex compute-memory interaction patterns. Additionally, it automatically lowers high-level tensor subprograms into optimized fused kernels, reducing the need for manual kernel engineering. Experimental results show that the fused kernels generated by ComFuse outperform those produced by TorchInductor across post-norm workloads and various complex computation scenarios, while supporting more flexible fusion patterns.
