A Survey of Bugs in AI-Generated Code

2025-12-04

AI Summary

AI-generated code suffers from diversity deficiencies, yet systematic error taxonomies, cross-model distribution patterns, and root-cause analyses remain lacking—hindering trustworthiness and maintainability. To address this, we first establish a comprehensive defect classification framework covering mainstream code generation models, derived from literature synthesis and qualitative analysis; it identifies six core bug categories (e.g., logical errors, API misuse, boundary condition omissions), characterizing their type distributions, severity gradients, and model-specific dependencies. Second, we construct a cross-model defect pattern association graph to reveal shared and divergent failure modes. Third, we propose a hierarchical repair strategy framework encompassing prompt engineering, post-hoc verification, and tailored evaluation metrics. Our work provides a reusable theoretical foundation and practical guidelines for quality assessment, iterative model improvement, and industrial deployment of AI programming tools.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Reasoning under Uncertainty: Probabilistic Programming
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Developers are widely using AI code-generation models, aiming to increase productivity and efficiency. However, there are also quality concerns regarding the AI-generated code. The generated code is produced by models trained on publicly available code, which are known to contain bugs and quality issues. Those issues can cause trust and maintenance challenges during the development process. Several quality issues associated with AI-generated code have been reported, including bugs and defects. However, these findings are often scattered and lack a systematic summary. A comprehensive review is currently lacking to reveal the types and distribution of these errors, possible remediation strategies, as well as their correlation with the specific models. In this paper, we systematically analyze the existing AI-generated code literature to establish an overall understanding of bugs and defects in generated code, providing a reference for future model improvement and quality assessment. We aim to understand the nature and extent of bugs in AI-generated code, and provide a classification of bug types and patterns present in code generated by different models. We also discuss possible fixes and mitigation strategies adopted to eliminate bugs from the generated code.

Problem

• Surveying bug types in AI-generated code
• Analyzing bug distribution across different AI models
• Proposing fixes for AI-generated code bugs

Innovation

• Systematically analyzing AI-generated code literature
• Classifying bug types and patterns in models
• Discussing fixes and mitigation strategies