Improving Compiler Bug Isolation by Leveraging Large Language Models

2025-06-21

AI Summary

Compiler bug localization suffers from poor scalability and high computational overhead due to expensive test-suite mutation. This paper proposes AutoCBI—the first approach to leverage large language models (LLMs) for compiler bug isolation. AutoCBI automatically summarizes the functional semantics of source files using an LLM, then fuses multi-dimensional evidence—including test coverage, compilation configurations, and error logs—via a customized prompt engineering framework to rank suspicious files. By eliminating the need for traditional mutation-based testing, AutoCBI significantly reduces resource consumption. Evaluated on 120 real-world bugs across GCC and LLVM, AutoCBI achieves Top-1 localization accuracy improvements of 66.7% and 340% over the state-of-the-art methods, respectively. The results demonstrate substantial gains in both precision and efficiency for bug localization in large-scale compiler scenarios.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Compilers play a foundational role in building reliable software systems, and bugs within them can lead to catastrophic consequences. The compilation process typically involves hundreds of files, making traditional automated bug isolation techniques inapplicable due to scalability or effectiveness issues. Current mainstream compiler bug localization techniques have limitations in test program mutation and resource consumption. Inspired by the recent advances of pre-trained Large Language Models (LLMs), we propose an innovative approach named AutoCBI, which (1) uses LLMs to summarize compiler file functions and (2) employs specialized prompts to guide LLM in reordering suspicious file rankings. This approach leverages four types of information: the failing test program, source file function summaries, lists of suspicious files identified through analyzing test coverage, as well as compilation configurations with related output messages, resulting in a refined ranking of suspicious files. Our evaluation of AutoCBI against state-of-the-art approaches (DiWi, RecBi and FuseFL) on 120 real-world bugs from the widely-used GCC and LLVM compilers demonstrates its effectiveness. Specifically, AutoCBI isolates 66.67%/69.23%, 300%/340%, and 100%/57.14% more bugs than RecBi, DiWi, and FuseFL, respectively, in the Top-1 ranked results for GCC/LLVM. Additionally, the ablation study underscores the significance of each component in our approach.

Problem

• Enhancing compiler bug isolation using LLMs
• Addressing scalability in automated bug localization
• Improving accuracy in ranking suspicious compiler files

Innovation

• Uses LLMs to summarize compiler file functions
• Employs specialized prompts for file reordering
• Leverages test programs and coverage data