Will It Survive? Deciphering the Fate of AI-Generated Code in Open Source

2026-01-23

AI Summary

This study challenges the prevailing assumption that AI-generated code is suitable only for one-off use by presenting the first large-scale empirical investigation into its long-term maintainability. Analyzing over 200,000 code units from 201 open-source projects, the authors employ survival analysis, textual feature modeling, and statistical tests—including Cramér’s V and hazard ratios (HR)—to compare modification rates and persistence durations between AI- and human-authored code. Results reveal that AI-generated code exhibits a 15.8 percentage point lower line-level modification rate, significantly reduced modification risk (HR = 0.842, p < 0.001), and longer persistence in codebases. The primary bottleneck in its evolution stems from organizational practices rather than intrinsic generation quality, with corrective modifications slightly higher yet exhibiting negligible effect size. These findings provide crucial empirical support for the sustainable integration of AI-generated code in software development.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Other Foundations of Human Computation & AI
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

The integration of AI agents as coding assistants into software development has raised questions about the long-term viability of AI agent-generated code. A prevailing hypothesis within the software engineering community suggests this code is"disposable", meaning it is merged quickly but discarded shortly thereafter. If true, organizations risk shifting maintenance burden from generation to post-deployment remediation. We investigate this hypothesis through survival analysis of 201 open-source projects, tracking over 200,000 code units authored by AI agents versus humans. Contrary to the disposable code narrative, agent-authored code survives significantly longer: at the line level, it exhibits a 15.8 percentage-point lower modification rate and 16% lower hazard of modification (HR = 0.842, p<0.001). However, modification profiles differ. Agent-authored code shows modestly elevated corrective rates (26.3% vs. 23.0%), while human code shows higher adaptive rates. However, the effect sizes are small (Cram\'er's V = 0.116), and per-agent variation exceeds the agent-human gap. Turning to prediction, textual features can identify modification-prone code (AUC-ROC = 0.671), but predicting when modifications occur remains challenging (Macro F1 = 0.285), suggesting timing depends on external organizational dynamics. The bottleneck for agent-generated code may not be generation quality, but the organizational practices that govern its long-term evolution.

Problem

• AI-generated code
• code survivability
• open source
• software maintenance
• disposable code

Innovation

• AI-generated code
• survival analysis
• code maintainability
• software evolution
• empirical software engineering