Insights into Security-Related AI-Generated Pull Requests

2026-04-21

AI Summary

This study investigates the security implications of AI-powered coding agents, which, while enhancing developer productivity, may introduce vulnerabilities and exhibit distinct code review behaviors compared to human developers. Analyzing 675 security-related pull requests (PRs) drawn from over 33,000 AI-generated PRs, the work employs large-scale data mining, vulnerability categorization, and qualitative content analysis to characterize recurrent defect patterns in AI-generated secure code—such as inefficient regular expressions, injection flaws, and path traversal vulnerabilities. It extends existing PR rejection taxonomies to accommodate AI contributions and uncovers the significant influence of procedural and social factors on the acceptance of AI-generated code. Notably, many defective PRs are still merged, with rejections primarily driven by process-related issues rather than code quality, and submission metadata shows limited impact on acceptance rates.

Technology Category

• Philosophy and Ethics of AI: Privacy & Security
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Recent years have experienced growing contributions of AI coding agents that assist human developers in various software engineering tasks. However, this growing AI-assisted autonomy raises questions about security and trust. In this paper, we analyze more than 33,000 AI-generated pull requests (PRs) and identify 675 security-related submissions made by agentic AIs. Then we examine the security-related PRs with a focus on recurring security weaknesses, review outcomes and latency, commit message quality, and rejection reasons. The results show that security-related AI PRs introduce a small set of recurring weaknesses such as regex inefficiencies, injection flaws, and path traversal. Many flawed contributions are still merged, while rejections often arise from social or process factors such as inactivity or missing test coverage. The commit message quality of AI PRs has a limited effect on acceptance or latency, in contrast to human PRs reported in previous studies. We also extend existing rejection taxonomies by adding categories that are unique to AI-generated security contributions. These findings offer new insights into the strengths and shortcomings of autonomous coding systems in secure software development.

Problem

• AI-generated code
• security vulnerabilities
• pull requests
• software security
• code review

Innovation

• AI-generated pull requests
• security weaknesses
• rejection taxonomy
• autonomous coding agents
• commit message quality