Problem
Research questions and friction points this paper is trying to address.
AI-generated issue reports
actionability
reproducibility
automated bug detection
trust erosion
LLM-Guided Issue Generation from Uncovered Code Segments
career value
170K/year
🤖 AI Summary
This work addresses the limited trust developers place in AI-generated bug reports due to their frequent lack of actionability and reproducibility. The authors propose a novel approach that integrates code coverage analysis with large language models (LLMs) to automatically detect defects in uncovered code regions and generate structured bug reports containing severity ratings, reproduction steps, and repair suggestions. A key innovation is an LLM-driven prioritization mechanism that substantially outperforms traditional rule-based methods. Evaluated on 13 Python projects, the method produced 10,467 reports; manual assessment of the top 130 revealed an 84.6% validity rate. Compared to CoverUp, it achieves higher defect validity (81.0% vs. 76.2%), a 50% improvement in P@3, and a 41% gain in mean reciprocal rank (MRR).
📝 Abstract
Developers are increasingly overwhelmed by AI-generated issue reports that lack actionability and reproducibility, eroding trust in automated bug detection tools. In this paper, we present IssueSpecter, an automated tool that finds bugs in uncovered code segments and automatically generates prioritized, actionable issue reports. IssueSpecter combines coverage analysis with LLM-based defect identification, producing structured reports complete with severity ratings, reproduction steps, and suggested fixes. We evaluate IssueSpecter on 13 actively maintained Python projects, generating 10,467 issue reports. Manual annotation of the top-130 ranked issues by IssueSpecter confirms that 84.6% of the LLM-generated issues are valid or warrant further investigation, with only 15.4% false positives. LLM-based ranking outperforms rule-based ranking by 50% at P@3 and 41% in MRR. The identified bugs cover a wide variety of types, from logic and boundary errors to security vulnerabilities and state consistency bugs. By ranking issues by priority, IssueSpecter aims to help developers focus their attention on the most impactful bugs first. Finally, we validate IssueSpecter through case studies reproducing real bugs surfaced from its generated issue reports, demonstrating its practical value for automatic bug discovery in open-source Python projects. Compared against CoverUp, a state-of-the-art coverage-driven test generation tool, IssueSpecter achieves a higher bug validity rate (81.0% vs. 76.2%) under identical evaluation conditions, using the same model and the same number of evaluated artifacts per project, while additionally providing structured issue reports with reproduction steps and candidate fixes that are immediately actionable without requiring developers to interpret generated test intent.
Innovation
Methods, ideas, or system contributions that make the work stand out.
LLM-guided bug detection
coverage analysis
actionable issue generation
automated bug reporting
structured issue reports
🔎 Similar Papers
No similar papers found.
