On the Role of Fault Localization Context for LLM-Based Program Repair

2026-04-07

AI Summary

This study investigates the impact of granularity and scope of fault localization context on the effectiveness of large language model (LLM)-driven automated program repair. Based on 500 SWE-bench Verified instances, the authors systematically evaluate 61 context configurations spanning file-level, element-level, and line-level granularities, finding that increased context does not necessarily improve performance. They propose an efficient strategy that integrates high-level semantic understanding with precise line-level fault localization and compare LLM-based retrieval against structure-guided heuristics on GPT-5-mini. Results show that file-level localization yields a 15–17× speedup, peak repair success is achieved with 6–10 relevant files, and LLM-based retrieval consistently outperforms heuristic methods in both effectiveness and resource efficiency.

Abstract

Fault Localization (FL) is a key component of Large Language Model (LLM)-based Automated Program Repair (APR), yet its impact remains underexplored. In particular, it is unclear how much localization is needed, whether additional context beyond the predicted buggy location is beneficial, and how such context should be retrieved. We conduct a large-scale empirical study on 500 SWE-bench Verified instances using GPT-5-mini, evaluating 61 configurations that vary file-level, element-level, and line-level context. Our results show that more context does not consistently improve repair performance. File-level localization is the dominant factor, yielding a 15-17x improvement over a no-file baseline. Expanding file context is often associated with improved performance, with successful repairs most commonly observed in configurations with approximately 6-10 relevant files. Element-level context expansion provides conditional gains that depend strongly on the file context quality, while line-level context expansion frequently degrades performance due to noise amplification. LLM-based retrieval generally outperforms structural heuristics while using fewer files and tokens. Overall, the most effective FL context strategy typically combines a broad semantic understanding at higher abstraction levels with precise line-level localization. These findings challenge our assumption that increasing the localization context uniformly improves APR, and provide practical guidance for designing LLM-based FL strategies.

Problem

• Fault Localization
• LLM-based Program Repair
• Context Retrieval
• Automated Program Repair
• Localization Context

Innovation

• Fault Localization
• Large Language Models
• Automated Program Repair
• Context Retrieval
• Empirical Study