Search papers, labs, and topics across Lattice.
This paper introduces a fast tile-level locality simulator designed to optimize the design space for general matrix multiplication (GEMM) on multi-chiplet GPUs, focusing on minimizing remote HBM traffic. By evaluating various kernel configurations, the simulator reveals that remote traffic can vary dramatically鈥攗p to 90x鈥攁cross different strategies for the same GEMM dimensions. Notably, it enables an AI agent to discover that a 2D block-swizzle CTA traversal significantly reduces remote traffic by up to 5.1 times compared to the best 1D traversal, highlighting the importance of CTA traversal order in GEMM performance optimization.
Remote HBM traffic can vary by 90x for the same GEMM dimensions, and a 2D block-swizzle traversal can slash this traffic by over 5 times.
Multi-chiplet GPUs split memory into local and remote HBM regions across a silicon interposer, and reducing the remote HBM traffic is crucial for the performance and energy efficiency of multi-chiplet GPUs. For general matrix multiplication (GEMM), the dominant operator in large language models (LLMs), the resulting inter-chiplet traffic depends strongly on kernel choices such as operand layout, CTA traversal order, and data placement, and the optimal strategy to minimize remote accesses is nontrivial. We present a fast, functional, tile-level locality simulator that models CTA scheduling, per-chiplet L2 caches, and local/remote HBM accesses to evaluate a full-size LLM GEMM configuration. Across representative LLM GEMMs, the simulator shows that remote traffic varies by up to 90x across the design space for the same GEMM dimensions. Moreover, using the simulator as feedback, an agentic AI discovers that a 2D block-swizzle CTA traversal reduces remote traffic over the best 1D traversal by up to 5.1x under round-robin placement, identifying CTA traversal order as a first-order, GEMM-dependent design knob for inter-chiplet traffic.
