Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a methodology for linking high-resolution power measurements of generative AI workloads to whole-facility energy demand in data centers. They measured power consumption at 0.1-second resolution for AI training, fine-tuning, and inference jobs using NVIDIA H100 GPUs, characterized by MLCommons and vLLM benchmarks. By scaling these power profiles using a bottom-up, event-driven data center energy model, the authors generate realistic whole-facility energy profiles that capture temporal fluctuations driven by AI workloads.
Publicly available, high-resolution power profiles of generative AI workloads running on H100 GPUs offer unprecedented insight into data center energy consumption, enabling more accurate infrastructure planning.
The rapid growth of generative artificial intelligence (AI) has introduced unprecedented computational demands, driving significant increases in the energy footprint of data centers. However, existing power consumption data is largely proprietary and reported at varying resolutions, creating challenges for estimating whole-facility energy use and planning infrastructure. In this work, we present a methodology that bridges this gap by linking high-resolution workload power measurements to whole-facility energy demand. Using NLR's high-performance computing data center equipped with NVIDIA H100 GPUs, we measure power consumption of AI workloads at 0.1-second resolution for AI training, fine-tuning and inference jobs. Workloads are characterized using MLCommons benchmarks for model training and fine-tuning, and vLLM benchmarks for inference, enabling reproducible and standardized workload profiling. The dataset of power consumption profiles is made publicly available. These power profiles are then scaled to the whole-facility-level using a bottom-up, event-driven, data center energy model. The resulting whole-facility energy profiles capture realistic temporal fluctuations driven by AI workloads and user-behavior, and can be used to inform infrastructure planning for grid connection, on-site energy generation, and distributed microgrids.