What's Wrong with Your Code Generated by Large Language Models? An Extensive Study

2024-07-08

arXiv.org

AI Summary

This study systematically uncovers a core limitation of large language models (LLMs) in complex code generation: their tendency to produce short yet highly cyclomatic-complex, error-prone code, with substantial discrepancies in defect distribution between real-world scenarios and standard benchmarks. Method: We propose the first fine-grained, three-level, 12-category taxonomy for LLM-generated code defects; and design a compiler-feedback-driven, training-free self-critique iterative correction framework, grounded in human annotation and empirical analysis (e.g., code length, cyclomatic complexity, API call statistics). Contribution/Results: Experiments show that two iterations of our method improve code pass rate by 29.2%. Crucially, we identify—for the first time—the sharp decline in LLM success rates on complex tasks and its correlation with structural defects. Our work establishes both theoretical foundations and practical tools for reliability assessment and improvement of LLM-based code generation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The increasing development of large language models (LLMs) in code generation has drawn significant attention among researchers. To enhance LLM-based code generation ability, current efforts are predominantly directed towards collecting high-quality datasets and leveraging diverse training technologies. However, there is a notable lack of comprehensive studies examining the limitations and boundaries of these existing methods. To bridge this gap, we conducted an extensive empirical study evaluating the performance of three leading closed-source LLMs and four popular open-source LLMs on three commonly used benchmarks. Our investigation, which evaluated the length, cyclomatic complexity and API number of the generated code, revealed that these LLMs face challenges in generating successful code for more complex problems, and tend to produce code that is shorter yet more complicated as compared to canonical solutions. Additionally, we developed a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types. Furthermore, to better understand the performance of LLMs in real-world projects, we manually created a real-world benchmark comprising 140 code generation tasks. Our analysis highlights distinct differences in bug distributions between actual scenarios and existing benchmarks. Finally, we propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback. Experimental results demonstrate that our approach can significantly mitigate bugs and increase the passing rate by 29.2% after two iterations, indicating substantial potential for LLMs to handle more complex problems.

Problem

• Evaluating limitations of LLMs in generating complex code solutions
• Analyzing bug patterns and root causes in LLM-generated code
• Developing methods to improve code quality through self-critique

Innovation

• Evaluated LLMs on code length and complexity metrics
• Developed taxonomy of bugs in generated code
• Proposed training-free iterative self-critique correction method