Anchored Self-Play for Code Repair

2026-07-03

AI Summary

This work addresses the scarcity of code repair data, which limits the performance of language models in real-world scenarios. To overcome this challenge, the authors propose the Anchored Self-Play (ASP) framework, wherein a single model simultaneously acts as both generator and corrector through a self-play mechanism augmented with reinforcement learning and automated curriculum learning. To prevent the self-play process from degenerating into unrealistic errors, ASP incorporates a reward signal based on embedding similarity to reference code, alongside a hybrid training strategy. Evaluated on the BugSourceBench benchmark, ASP achieves an absolute improvement of 7.0 percentage points (a 24% relative gain) in average repair rate over standard self-play, demonstrating state-of-the-art performance across diverse error types generated by both humans and models.

Abstract

Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug--fix tasks. We propose __generator--fixer self-play__, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduce BugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. On BugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We propose Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by $+24\%$ relative / $+7.0$ pp absolute, with gains on bugs from both LMs and humans.

Problem

• code repair
• self-play
• language models
• bug generation
• generalization

Innovation

• Anchored Self-Play
• code repair
• self-play curriculum
• bug generation
• code embedding