Search papers, labs, and topics across Lattice.
This study evaluates Model FLOPs Utilization (MFU) as a software-defined predictor of GPU power consumption during LLM training, addressing the limitations of existing power models that rely on hardware utilization counters. By benchmarking nearly 3000 training runs across six different GPUs and various configurations, the authors demonstrate that a linear MFU-based power model can accurately predict GPU power usage when workloads are compute-bound. The results reveal that tailoring the model to specific GPU configurations significantly reduces prediction error from approximately 10% to about 1%, aligning closely with measurement noise levels.
MFU can predict GPU power consumption with remarkable accuracy, achieving a 1% error rate in compute-bound LLM training scenarios.
High-fidelity performance simulators are essential for designing and configuring efficient AI systems, yet today's tools lack the ability to predict power consumption. Established GPU power models rely on hardware utilization counters, which do not exist until the workload has actually run. This work evaluates whether Model FLOPs Utilization (MFU)-an analytical, software-defined metric relating achieved throughput to peak hardware capability-can serve as a portable, software-defined predictor of GPU power for LLMs. We benchmark almost 3000 single-device training runs across six GPUs, covering different model families, numerical precisions, batch sizes, and context-window lengths. We find that a linear MFU-based power model fits every tested GPU as long as the workload is compute-bound, as in production LLM training. Fitting per-(GPU, dtype, batch size) instead of per-GPU drops the within-cell mean error from around 10% to around 1%, matching the cross-repeat measurement-noise floor.
