SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests

2026-07-01

AI Summary

This work addresses the limitations of current large language model (LLM)-based program repair approaches, which struggle to effectively leverage bug-reproducing tests (BRTs) and are often misled by incomplete or unreliable test cases, resulting in incomplete or incorrect patches. To overcome this, the authors propose SWE-Doctor, a novel method that integrates multi-perspective BRT generation with runtime debugging diagnostics. By executing diverse test cases to construct diagnostic traces and incorporating fault localization information, SWE-Doctor guides the LLM to produce more accurate and complete repairs. This approach moves beyond the conventional use of BRTs solely for validation, achieving state-of-the-art results with repair success rates of 75.7% on SWE-bench Verified and 59.4% on SWE-bench Pro—improvements of 8.0 to 8.9 percentage points over existing baselines.

Abstract

Large language model (LLM)-based software engineering agents are increasingly developed to resolve software issues by generating patches from issue reports and code repositories. Bug reproduction tests (BRTs) are an important building block for such agents and have been shown useful for patch validation. However, it remains unclear whether BRTs can also help the more central stage of patch generation. We first conduct a preliminary study and find that directly using advanced BRT generators to guide patch generation is not beneficial: fail-to-fail BRTs can mislead agents, while even fail-to-pass BRTs bring limited or negative gains. Our analysis reveals two reasons: fail-to-pass BRTs may cover only one manifestation of the reported issue, leading to partial patches, whereas fail-to-fail BRTs are unreliable as direct patch-generation targets. Motivated by these insights, we propose SWE-Doctor, a software issue resolution agent that guides patch generation with runtime diagnoses derived from multi-faceted BRT executions. SWE-Doctor first generates multi-faceted BRTs for different behavioral requirements stated in the issue, then executes and debugs these BRTs to construct runtime-grounded diagnosis records, and finally uses the diagnoses together with localization information inferred during BRT generation to guide patch generation and reduce partial patches. We evaluate SWE-Doctor on Python bug-fixing issues from the widely adopted SWE-bench Verified and SWE-bench Pro across five LLM backends. SWE-Doctor consistently outperforms existing agents across all 10 LLM-benchmark combinations, achieving average resolution rates of 75.7% on SWE-bench Verified and 59.4% on SWE-bench Pro. In particular, on the more challenging SWE-bench Pro, SWE-Doctor improves the average resolution rate by 8.0-8.9 percentage points over the baseline agents.

Problem

• Bug Reproduction Tests
• Patch Generation
• Software Engineering Agents
• LLM-based Repair
• Runtime Diagnosis

Innovation

• bug reproduction tests
• runtime diagnosis
• patch generation
• software engineering agents
• multi-faceted testing