EMO: Energy Efficiency Modeling and Optimization for AI Workloads

2026-07-14

AI Summary

This work addresses the challenge of high energy consumption in GPU-accelerated AI workloads, where existing approaches struggle to achieve fine-grained and general-purpose energy efficiency without compromising end-to-end latency. The authors propose a lightweight optimization framework that constructs an underlying dependency graph to identify slack windows introduced by asynchronous execution. By integrating a dependency-aware kernel packing strategy, the framework preserves the critical path while building a high-fidelity, low-overhead energy-latency model. Energy efficiency is then formulated as a constrained combinatorial optimization problem. This approach uniquely balances workload generality with fine-grained slack exploitation, achieving 15%–28% energy savings with only 2%–5% performance degradation and negligible optimization overhead.

Abstract

The massive energy consumption of GPU-accelerated AI workloads challenges sustainable computing. We observe that execution asynchrony (e.g., CPU-GPU, concurrent streams, multi-GPU) creates slack, allowing non-critical kernels to run at lower frequencies to save energy without impacting end-to-end latency. However, existing approaches fail to simultaneously achieve workload generality and fine-grained slack discovery, while high-fidelity modeling incurs prohibitive overhead. We present EMO, a lightweight framework exploiting these fine-grained opportunities. First, to identify where to optimize, EMO constructs a low-level dependency graph capturing asynchrony and performs what-if timing analysis to precisely identify slack windows. Second, to determine how to optimize, EMO introduces dependency-aware kernel packing. It aggregates kernels to preserve critical paths while collapsing redundant details, enabling high-fidelity latency-energy modeling with minimal profiling cost. Finally, EMO combines graph analysis and pack-level models to formulate energy optimization as a constrained combinatorial problem, efficiently solving for optimal frequency policies under given latency targets. Evaluations show EMO reduces energy consumption by 15%--28% with only 2%--5% performance loss and negligible overhead.

Problem

• energy efficiency
• AI workloads
• execution asynchrony
• slack discovery
• GPU computing

Innovation

• energy efficiency
• asynchronous execution
• dependency-aware kernel packing
• slack-aware optimization
• frequency scaling