A Survey On Large Language Models For Code Generation

2025-03-03

AI Summary

This paper systematically reviews bottlenecks, optimization strategies, and evaluation frameworks for large language models (LLMs) in code generation. It addresses key limitations in functional correctness, readability, and security. To overcome these, the paper unifies major fine-tuning paradigms—including supervised fine-tuning, instruction tuning, reinforcement learning from human feedback (RLHF), and code-specific pretraining—and proposes a multi-granularity evaluation framework covering syntax, semantics, security, and engineering best practices. Its core contribution is a novel “Capability–Optimization–Evaluation–Deployment” full-stack analytical model that, for the first time, coherently maps the fine-tuning methodology spectrum to industrial tools (e.g., CodeLlama, GitHub Copilot). The work clarifies current technical boundaries, distills reusable methodological guidelines, and provides both theoretical foundations and practical pathways to enable low-barrier, cross-domain programming democratization. (149 words)

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large Language Models (LLMs) have demonstrated their remarkable capabilities in numerous fields. This survey focuses on how LLMs empower users, regardless of their technical background, to use human languages to automatically generate executable code. We begin with understanding LLMs' limitations and challenges in automated code generation. Subsequently, we review various fine-tuning techniques designed to enhance both the performance and adaptability of LLMs in code generation tasks. We then review the existing metrics and benchmarks for evaluations to assess model performance based on fine-tuning techniques. Finally, we explore the applications of LLMs (e.g. CodeLlama, GitHub Copilot, ToolGen) in code generation tasks to illustrate their roles and functionalities. This survey provides a comprehensive overview of LLMs for code generation, helps researchers in diverse fields better understand the current state-of-the-art technologies, and offers the potential of effectively leveraging LLMs for code generation tasks.

Problem

• LLMs enable non-technical users to generate code using human language.
• Survey explores fine-tuning techniques to improve LLMs' code generation performance.
• Review of metrics, benchmarks, and applications of LLMs in code generation.

Innovation

• Fine-tuning LLMs for enhanced code generation performance
• Using human language to generate executable code automatically
• Evaluating LLMs with metrics and benchmarks post fine-tuning