SHERLOC: Structured Diagnostic Localization for Code Repair Agents

2026-06-23

AI Summary

This work addresses the high computational cost and insufficient diagnostic context in existing code repair agents during fault localization. It proposes a training-free, structured localization framework that reframes fault localization from file retrieval to an actionable diagnostic task. By integrating hypothesis-driven exploration from large reasoning models, lightweight code repository tools, and a self-recovery mechanism, the approach generates structured diagnostic information. Evaluated on SWE-Bench Lite and Verified, it achieves 84.33% precision and 81.27% recall, respectively. When injected into downstream repair pipelines, its localization outputs improve repair success rates by an average of 5.95 percentage points while reducing localization and overall token consumption by 36.7% and 23.1%, respectively.

Abstract

LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have emerged, yet are still evaluated as file retrieval rather than actionable diagnosis, producing locations without the diagnostic context a repair agent needs. We introduce SHERLOC (Structured Hypothesis-driven Exploration and Reasoning for Localization), a training-free framework pairing a reasoning LLM with compact repository tools and self-recovery, without fine-tuning or multi-agent orchestration. SHERLOC reaches state-of-the-art localization across model scales: 84.33% accuracy@1 on SWE-Bench Lite and 81.27% recall@1 on SWE-Bench Verified; at ~30B parameters, it matches or outperforms other agentic methods. Injecting our locations and diagnostic findings into repair agents yields, on average, +5.95 pp resolve rate on SWE-Bench Verified while cutting localization and total tokens by 36.7% and 23.1%.

Problem

• code repair
• fault localization
• diagnostic context
• repository-level coding
• LLM agents

Innovation

• structured localization
• diagnostic reasoning
• training-free framework
• code repair agents
• repository-level debugging