Factors Influencing the Quality of AI-Generated Code: A Synthesis of Empirical Evidence

2026-03-26

AI Summary

This study addresses the growing concerns regarding the quality, reliability, and security of AI-generated code by systematically identifying its key influencing factors. Through a rigorous systematic literature review—combining AI-assisted screening with manual validation—the authors synthesize evidence from 24 empirical studies to integrate, for the first time, three critical dimensions: human factors (e.g., developer expertise), system characteristics (e.g., prompt design), and human-AI interaction aspects (e.g., task specification). The findings reveal that AI-assisted programming functions as a socio-technical system, wherein different quality attributes—such as correctness and security—exhibit markedly divergent behaviors. While the approach demonstrates considerable potential for enhancing software development, it also introduces non-trivial risks, thereby offering both theoretical insights and practical guidance for the future design and evaluation of AI-powered programming tools.

Technology Category

• Humans and AI: Other Foundations of Human Computation & AI
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Context: The rapid adoption of AI-assisted code generation tools, such as large language models (LLMs), is transforming software development practices. While these tools promise significant productivity gains, concerns regarding the quality, reliability, and security of AI-generated code are increasingly reported in both academia and industry. --Objective: This study aims to systematically synthesize existing empirical evidence on the factors influencing the quality of AI-generated source code and to analyze how these factors impact software quality outcomes across different evaluation contexts. --Method: We conducted a systematic literature review (SLR) following established guidelines, supported by an AI-assisted workflow with human oversight. A total of 24 primary studies were selected through a structured search and screening process across major digital libraries. Data were extracted and analyzed using qualitative, pattern-based evidence synthesis. --Results: The findings reveal that code quality in AI-assisted development is influenced by a combination of human factors, AI system characteristics, and human AI interaction dynamics. Key influencing factors include prompt design, task specification, and developer expertise. The results also show variability in quality outcomes such as correctness, security, maintainability, and complexity across studies, with both improvements and risks reported. --Conclusion: AI-assisted code generation represents a socio-technical shift in software engineering, where achieving high-quality outcomes depends on both technological and human factors. While promising, AI-generated code requires careful validation and integration into development workflows.

Problem

• AI-generated code
• code quality
• software engineering
• large language models
• empirical evidence

Innovation

• systematic literature review
• AI-assisted code generation
• code quality
• human-AI interaction
• prompt design