A^2Agent: Action-Aware Reinforcement Learning for Repository-Level Code Localization Agents

2026-08-30

AI Summary

为解决现有方法无法有效识别每步动作有效性的问题，提出了一种结合每步奖励序列和动作级优势估计方案的动作感知强化学习方法，以改进代码定位。

Abstract

Localizing issue-relevant code regions is a critical step in automated software engineering. However, due to their reliance on sparse trajectory-level signals, existing methods cannot identify which per-turn actions are effective and often discover correct code regions during exploration but fail to commit them. To address these limitations, we propose an action-aware reinforcement learning method that combines a per-turn reward sequence rewarding both the discovery and commitment of gold code regions with an action-level advantage estimation scheme that isolates each action's credit by grouping turns sharing the same exploration context. Extensive evaluations show that our method improves the average F1 over the state-of-the-art (SOTA) by 1.58% on SWE-Bench Verified and 8.55% on SWE-Bench Pro, with our 4B model outperforming baselines up to 8x larger. Our code is available at https://github.com/donian00/A2Agent.

Problem

• code localization
• reinforcement learning
• sparse trajectory-level signals
• action-aware

Innovation

• action-aware reinforcement learning
• per-turn reward sequence
• action-level advantage estimation