Search papers, labs, and topics across Lattice.
This paper introduces EMO, a lightweight framework designed to optimize energy consumption in GPU-accelerated AI workloads by leveraging execution asynchrony to identify and exploit slack. EMO constructs a low-level dependency graph to perform what-if timing analysis, allowing for precise identification of slack windows, and employs dependency-aware kernel packing to enable high-fidelity latency-energy modeling with minimal profiling overhead. The results demonstrate that EMO can reduce energy consumption by 15% to 28% while maintaining a performance loss of only 2% to 5%, showcasing its effectiveness in achieving energy efficiency without significant trade-offs in latency.
EMO achieves up to 28% energy savings in AI workloads with minimal performance impact by intelligently exploiting execution asynchrony.
The massive energy consumption of GPU-accelerated AI workloads challenges sustainable computing. We observe that execution asynchrony (e.g., CPU-GPU, concurrent streams, multi-GPU) creates slack, allowing non-critical kernels to run at lower frequencies to save energy without impacting end-to-end latency. However, existing approaches fail to simultaneously achieve workload generality and fine-grained slack discovery, while high-fidelity modeling incurs prohibitive overhead. We present EMO, a lightweight framework exploiting these fine-grained opportunities. First, to identify where to optimize, EMO constructs a low-level dependency graph capturing asynchrony and performs what-if timing analysis to precisely identify slack windows. Second, to determine how to optimize, EMO introduces dependency-aware kernel packing. It aggregates kernels to preserve critical paths while collapsing redundant details, enabling high-fidelity latency-energy modeling with minimal profiling cost. Finally, EMO combines graph analysis and pack-level models to formulate energy optimization as a constrained combinatorial problem, efficiently solving for optimal frequency policies under given latency targets. Evaluations show EMO reduces energy consumption by 15%--28% with only 2%--5% performance loss and negligible overhead.
