GREPO: A Benchmark for Graph Neural Networks on Repository-Level Bug Localization

2026-02-14

AI Summary

This work addresses the challenge that large language models, constrained by limited context windows, struggle to precisely localize bugs across entire code repositories. To this end, we propose GREPO, the first graph neural network (GNN) benchmark specifically designed for repository-level bug localization, encompassing 86 Python repositories and 47,294 bug-fixing tasks, along with structured graph data ready for GNN training. By constructing code dependency graphs and leveraging GNNs to model complex semantic relationships, our approach substantially outperforms conventional retrieval baselines—including keyword matching, text similarity, and breadth-first search—demonstrating the efficacy of GNNs in repository-scale bug localization and establishing a standardized evaluation resource previously lacking in this domain.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Repository-level bug localization-the task of identifying where code must be modified to fix a bug-is a critical software engineering challenge. Standard Large Language Modles (LLMs) are often unsuitable for this task due to context window limitations that prevent them from processing entire code repositories. As a result, various retrieval methods are commonly used, including keyword matching, text similarity, and simple graph-based heuristics such as Breadth-First Search. Graph Neural Networks (GNNs) offer a promising alternative due to their ability to model complex, repository-wide dependencies; however, their application has been hindered by the lack of a dedicated benchmark. To address this gap, we introduce GREPO, the first GNN benchmark for repository-scale bug localization tasks. GREPO comprises 86 Python repositories and 47294 bug-fixing tasks, providing graph-based data structures ready for direct GNN processing. Our evaluation of various GNN architectures shows outstanding performance compared to established information retrieval baselines. This work highlights the potential of GNNs for bug localization and established GREPO as a foundation resource for future research, The code is available at https://github.com/qingpingmo/GREPO.

Problem

• bug localization
• graph neural networks
• repository-level
• code repositories
• software engineering

Innovation

• Graph Neural Networks
• Bug Localization
• Repository-Level
• Benchmark
• Code Representation