Bug Fixing with Broader Context: Enhancing LLM-Based Program Repair via Layered Knowledge Injection

2025-06-30

AI Summary

Complex bugs—those resistant to repair using only local code and error information (e.g., stack traces)—pose significant challenges for automated program repair. Method: This paper proposes a hierarchical knowledge injection framework that systematically integrates multi-granularity, repository- or project-level contextual signals: function-level semantics, test cases, code dependencies, commit histories, documentation, and patches from previously fixed bugs. The framework uncovers bug-type–specific demands across context abstraction levels and enables large language models (LLMs), including Llama 3.3 and GPT-4o-mini, to adaptively perceive and leverage structured knowledge. Contribution/Results: Evaluated on BugsInPy, the framework achieves a 79% repair rate (250/314), outperforming prior state-of-the-art by 23%. Crucially, all bug categories exhibit statistically significant improvement with repository-level context. This work advances automated program repair from static, local reasoning toward interactive, context-aware, adaptive paradigms.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Prompting LLMs with bug-related context (e.g., error messages, stack traces) improves automated program repair, but many bugs still remain unresolved. In real-world projects, developers often rely on broader repository and project-level context beyond the local code to resolve such bugs. In this paper, we investigate how automatically extracting and providing such knowledge can improve LLM-based program repair. We propose a layered knowledge injection framework that incrementally augments LLMs with structured context. It starts with the Bug Knowledge Layer, which includes information such as the buggy function and failing tests; expands to the Repository Knowledge Layer, which adds structural dependencies, related files, and commit history; and finally injects the Project Knowledge Layer, which incorporates relevant details from documentation and previously fixed bugs. We evaluate this framework on a dataset of 314 bugs from BugsInPy using two LLMs (Llama 3.3 and GPT-4o-mini), and analyze fix rates across six bug types. By progressively injecting knowledge across layers, our approach achieves a fix rate of 79% (250/314) using Llama 3.3, a significant improvement of 23% over previous work. All bug types show improvement with the addition of repository-level context, while only a subset benefit further from project-level knowledge, highlighting that different bug types require different levels of contextual information for effective repair. We also analyze the remaining unresolved bugs and find that more complex and structurally isolated bugs, such as Program Anomaly and GUI bugs, remain difficult even after injecting all available information. Our results show that layered context injection improves program repair and suggest the need for interactive and adaptive APR systems.

Problem

• Enhancing LLM-based program repair with broader contextual knowledge
• Improving bug fix rates via layered repository and project-level information
• Addressing unresolved complex bugs needing adaptive context injection

Innovation

• Layered knowledge injection framework for LLMs
• Bug, Repository, Project Knowledge Layers
• Improves fix rate by 23% over prior work