Smaller Models, Unexpected Costs: Trade-offs in LLM Quantization for Automated Program Repair

2026-06-25

AI Summary

This study systematically investigates the trade-offs between effectiveness and non-functional overhead—specifically memory consumption, energy usage, and inference latency—in quantized large language models for automated program repair. Evaluating 13 quantization configurations across six models on the HumanEval-Java and Defects4J benchmarks, the work incorporates hardware utilization and energy consumption into the assessment framework for the first time and employs Pareto front analysis. The findings reveal that quantization can reduce memory footprint by up to 85%, yet often increases inference latency and energy consumption. Notably, 48% of the configurations are strictly dominated, highlighting strong dependencies on model architecture and task complexity. Furthermore, quantized models fix a comparable number of bugs as their full-precision counterparts, with low overlap in repaired instances, suggesting significant complementary potential.

Abstract

Language Models (LLMs) are powerful toolsand have been increasingly adopted for complex software engineering tasks. As the number of parameters increases, results can often be improved, but this also imposes substantialmemory requirements. While quantization effectively reduces thememory footprint, its overall impact is often summarized onlyby benchmark scores, which mask changes in model behaviorand non-functional overheads. In this work, we conduct anempirical evaluation of LLM quantization using AutomatedProgram Repair (APR), a complex task in software engineering.We analyze 13 quantization configurations spanning differentbit-widths, methods, and target components (weights and KVcache) across six representative LLMs, evaluated on two APRbenchmarks (HumanEval-Java and Defects4J). Our findings reveal that base and quantized models can provide different sets of repaired problems with little overlap, whileretaining a comparable number of repaired problems. Althoughquantization successfully reduces memory footprints by up to85%, it increases both inference time and energy consumption,which we attribute to suboptimal hardware utilization. OurPareto trade-off analysis shows that 48% of the configurationsevaluated are strictly dominated by alternatives. Rather thanidentifying a superior quantization method, our findings highlightthat the trade-offs between effectiveness, memory footprint,and energy efficiency are sensitive to the underlying modelarchitecture and the complexity of the task.

Problem

• LLM Quantization
• Automated Program Repair
• Memory Footprint
• Energy Consumption
• Inference Time

Innovation

• LLM quantization
• Automated Program Repair
• memory-energy trade-off
• Pareto analysis
• non-functional overhead