Why Agentic-PRs Get Rejected: A Comparative Study of Coding Agents

2026-02-04

AI Summary

This study systematically investigates the reasons for rejection of pull requests generated by coding agents (Agentic-PRs) and their differences from human-authored PRs. Leveraging 654 rejected PRs from the AIDev dataset, we conduct a large-scale empirical analysis combined with qualitative coding to compare rejection patterns across five types of coding agents and human developers. Our analysis identifies seven failure modes unique to Agentic-PRs—such as Devin automatically retracting inactive PRs—and reveals that 67.9% of rejected PRs lack explicit reviewer feedback. To address this ambiguity, we propose a heuristic preprocessing method that effectively reduces the proportion of unclear rejections, offering a novel pathway toward improving the quality of agent-generated code contributions.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Agentic coding -- software development workflows in which autonomous coding agents plan, implement, and submit code changes with minimal human involvement -- is rapidly gaining traction. Prior work has shown that Pull Requests (PRs) produced using coding agents (Agentic-PRs) are accepted less often than PRs that are not labeled as agentic (Human-PRs). The rejection reasons for a single agent (Claude Code) have been explored, but a comparison of how rejection reasons differ between Agentic-PRs generated by different agents has not yet been performed. This comparison is important since different coding agents are often used for different purposes, which can lead to agent-specific failure patterns. In this paper, we inspect 654 rejected PRs from the AIDev dataset covering five coding agents, as well as a human baseline. Our results show that seven rejection modes occur only in Agentic-PRs, including distrust of AI-generated code. We also observe agent-specific patterns (e.g., automated withdrawal of inactive PRs by Devin), reflecting differences in how agents are configured and used in practice. Notably, a large proportion of rejected PRs (67.9%) lack explicit reviewer feedback, making their rejection reasons difficult to determine. To mitigate this issue, we propose a set of heuristics that reduce the proportion of such cases, offering a practical preprocessing step for future studies of PR rejection in agentic coding.

Problem

• Agentic-PRs
• Pull Request rejection
• coding agents
• rejection reasons
• AI-generated code

Innovation

• agentic coding
• pull request rejection
• coding agents
• rejection heuristics
• AI-generated code