Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations

2026-07-10

AI Summary

This study addresses the unclear impact of key configuration choices—attention kernel type, prefix caching, and chunked prefill—on energy consumption, latency, and generation accuracy in the vLLM inference engine. Through 9,000 experiments across five open-source large language models and five task types (yielding 93,600 metrics), the work systematically evaluates the combinatorial effects of these settings. It reveals, for the first time, that inference configurations not only significantly affect efficiency but can also unexpectedly alter output accuracy. While model selection governs the overall performance–accuracy trade-off, careful configuration tuning enables local Pareto-optimal improvements. The findings show that attention kernel and prefix caching substantially influence performance and energy use, whereas chunked prefill offers limited benefits under default settings. Critically, all effects are highly model- and task-dependent, with no universally optimal configuration.

Abstract

Large Language Models are reshaping how software is developed and maintained. They are typically deployed in production using inference engines such as vLLM, which can efficiently serve pre-trained, highly configurable models. While prior work has focused on model architectures and hardware acceleration, the impact of inference engine configuration on energy consumption, performance, and output quality remains poorly understood. In this paper, we present a large-scale controlled study of three selected vLLM configuration options: attention kernel type, prefix caching, and chunked prefill. We evaluate all combinations of these configurations across 5 open-weight LLMs and 5 diverse inference tasks, totaling $9,000$ runs and $93,600$ measures. We analyze energy consumption, latency, and accuracy, and examine both main effects and interaction effects between configuration options and tasks. Our results show that the studied configuration options significantly impact energy and performance, mainly driven by attention type and prefix caching, while chunked prefill has a limited effect under the default vLLM serving configuration and evaluated workloads. These effects are highly model- and workload-dependent, and no configuration is universally optimal. We further show that model choice dominates global trade-offs, while configuration tuning provides local improvements along the Pareto frontier. Unexpectedly, inference options can also affect model accuracy.

Problem

• inference configuration
• energy consumption
• performance
• accuracy
• vLLM

Innovation

• vLLM configuration
• energy-performance trade-off
• attention kernel
• prefix caching
• inference accuracy