Search papers, labs, and topics across Lattice.
This paper introduces DGNA, a novel methodology that dissects the Non-Uniform Memory Access (NUMA) architecture of GPU memory hierarchies through microbenchmarking and data analysis. By measuring L2 cache and DRAM latencies without relying on architecture-specific instructions and employing a Gaussian mixture model to refine latency distributions, the authors reveal critical insights into the NUMA node architecture and memory allocation strategies of NVIDIA's A100 and H100 GPUs. The findings provide a foundational understanding that can optimize GPU applications and inform future architectural designs, marking a significant advancement in GPU memory efficiency research.
Unveiling the hidden NUMA architecture of GPUs could revolutionize how we optimize memory efficiency in high-performance computing.
Graphics Processing Units (GPUs), due to their immense parallel processing capabilities, have become essential across various fields, including gaming and artificial intelligence. With significant advancements in GPU cores, GPU memory efficiency has lagged, resulting in bottlenecks that can limit workload efficiency. To bridge this gap, a deep understanding of GPU memory architectures, particularly Non-Uniform Memory Access (NUMA) mechanisms within L2 and DRAM, is essential for optimizing applications, designing new architectures, and building accurate simulators. However, the latest GPU hardware from vendors like NVIDIA and AMD is still a black-box, making it challenging for researchers to understand the details of their design. In this paper, we introduce DGNA, a methodology designed to unveil the NUMA architecture of the GPU memory hierarchy through microbenchmarking and data analysis. Specifically, we propose an approach to measuring the latency of L2 caches and DRAM without relying on the intrinsic instructions of the architecture and apply a Gaussian mixture model to filter out outliers and accurately determine latency distributions. We apply DGNA on NVIDIA's A100 and H100 GPUs, revealing NUMA node architecture, SM-NUMA relationships, and NUMA-aware memory allocation strategies used to maintain cache coherence. To the best of our knowledge, this is the first paper to detail the NUMA architecture within the GPU memory subsystem.
NUS