PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

2026-05-20

AI Summary

This work addresses the high energy consumption of large language model (LLM) inference in data centers, where existing systems treat GPU power consumption as a static constraint, struggling to balance energy efficiency and quality of service (QoS). The paper introduces a novel approach that treats the GPU power limit as a primary control variable, jointly optimizing it with software-level parameters such as batch size. By integrating lightweight offline power-performance modeling with a feedback controller within the vLLM framework, the method enables dynamic power management without requiring model retraining or API modifications, supporting both dense and mixture-of-experts models. Evaluated on multi-GPU systems, the approach improves energy efficiency by up to 26.3%, reduces QoS violations by 4–7× under power constraints, and effectively tracks time-varying power budgets.

Abstract

Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.

Problem

• LLM inference
• power-aware serving
• Mixture-of-Experts
• energy efficiency
• GPU power management

Innovation

• Power-aware serving
• Mixture-of-Experts
• GPU power management
• Energy efficiency
• LLM inference