Search papers, labs, and topics across Lattice.
This paper introduces an AI benchmarking framework for analyzing the power-aware performance of vision and language models on modern GPUs. The framework evaluates throughput and energy efficiency under different power capping scenarios on NVIDIA H100/H200 and AMD MI300X GPUs. Results show that the optimal power cap varies significantly based on application type and GPU architecture, with NVIDIA GPUs exhibiting qualitatively different performance-energy trade-offs due to HBM configuration differences.
Forget one-size-fits-all power caps: the optimal energy efficiency for AI workloads on GPUs varies wildly by application and architecture.
Artificial Intelligence (AI) workloads drive a rapid expansion of high-performance computing (HPC) infrastructures and increase their power and energy demands towards a critical level. AI benchmarks representing state-of-the art workloads and their understanding in the context of performance-energy trade-offs are critical to deploy efficient infrastructures and can guide energy efficiency measures, such as power capping. We introduce a benchmarking framework with popular deep learning applications from computer vision (image classification and generation) and large language models (continued pre-training and inference) implementing modern methods. Our performance analysis focuses on throughput rather than time to"completion", which is the standard metric in HPC. We analyse performance and energy efficiency under various power capping scenarios on NVIDIA H100, NVIDIA H200, and AMD MI300X GPUs. Our results reveal that no universal optimal power cap exists, as the efficiency peak varies across application types and GPU architectures. Interestingly, the two NVIDIA GPUs which mainly differ in their HBM configuration show qualitatively different performance-energy trade-offs. The developed benchmarking framework will be released as a public tool.
