FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

2026-08-09

AI Summary

Traditional rejection fine-tuning (RFT) discards failed trajectories generated by code agents on verifiable software engineering tasks, thereby wasting valuable information embedded in hard negative examples. This work proposes FailForge, a framework that systematically transforms such failure trajectories into structured skills. By analyzing error feedback and diagnosing execution traces, FailForge distills actionable repair strategies and injects them into the training process via knowledge distillation. During inference, external prompts are removed to encourage the model to internalize these repair capabilities. Seamlessly integrated into RFT, FailForge improves the pass rate of Qwen3.5-4B by 6.6 percentage points on SWE-bench Verified, successfully recovering over 26% of previously failed instances and substantially enhancing performance on the most challenging problems.

Abstract

Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts. However, even strong code agents repeatedly fail on a substantial fraction of such tasks, and standard RFT simply discards these failures. The discarded samples are precisely the hardest and most informative ones, drawn from verifiable instances that are costly to curate. Stronger base models may reduce the number of failures, but the remaining hard cases still define the frontier for further improvement. We propose FailForge, an agentic framework that converts failed rollouts into training signal. For each failed instance, an agent diagnoses the failure from error feedback and execution traces, distills the diagnosis into a concise and actionable skill, and injects the skill into the agent context for a guided second attempt. Trajectories that succeed under skill guidance are folded back into the RFT corpus. Crucially, the skill is removed at training time, so the model internalizes the recovered behavior rather than relying on external hints at inference. FailForge recovers over 26% of previously failed instances at marginal additional cost, and training Qwen3.5-4B on the augmented corpus improves the SWE-bench Verified resolve rate by 6.6 points over a strong RFT baseline, with gains concentrated on the hardest problems.

Problem

• rejection sampling fine-tuning
• code agents
• failed trajectories
• procedural competence
• SWE-bench

Innovation

• FailForge
• rejection sampling fine-tuning
• failure distillation
• procedural competence
• code agents