A Survey of Reinforcement Learning for Software Engineering

2025-07-14

AI Summary

The application of reinforcement learning (RL) in software engineering lacks a systematic, comprehensive survey. Method: We conduct the first panoramic systematic literature review (SLR), covering 22 top-tier conferences and synthesizing 115 deep RL studies. Using multidimensional analysis—spanning algorithm types, datasets, model architectures, and evaluation methodologies—we develop a taxonomy organized along four software engineering activities: design, development, quality assurance, and maintenance. Contribution/Results: Our analysis reveals prevailing practices, shared limitations—including data scarcity, inconsistent evaluation protocols, and poor reproducibility—and emerging evolutionary trends. We propose actionable recommendations to address these challenges and publicly release all research artifacts, including the curated literature corpus, coding scheme, and analysis tools. This open resource establishes an extensible benchmarking framework and identifies concrete directions for future research in RL for software engineering.

Technology Category

• Machine Learning: Reinforcement Learning
• Application Domains: Software Engineering
• Knowledge Representation and Reasoning: Knowledge Engineering

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Reinforcement Learning (RL) has emerged as a powerful paradigm for sequential decision-making and has attracted growing interest across various domains, particularly following the advent of Deep Reinforcement Learning (DRL) in 2015. Simultaneously, the rapid advancement of Large Language Models (LLMs) has further fueled interest in integrating RL with LLMs to enable more adaptive and intelligent systems. In the field of software engineering (SE), the increasing complexity of systems and the rising demand for automation have motivated researchers to apply RL to a broad range of tasks, from software design and development to quality assurance and maintenance. Despite growing research in RL-for-SE, there remains a lack of a comprehensive and systematic survey of this evolving field. To address this gap, we reviewed 115 peer-reviewed studies published across 22 premier SE venues since the introduction of DRL. We conducted a comprehensive analysis of publication trends, categorized SE topics and RL algorithms, and examined key factors such as dataset usage, model design and optimization, and evaluation practices. Furthermore, we identified open challenges and proposed future research directions to guide and inspire ongoing work in this evolving area. To summarize, this survey offers the first systematic mapping of RL applications in software engineering, aiming to support both researchers and practitioners in navigating the current landscape and advancing the field. Our artifacts are publicly available: https://github.com/KaiWei-Lin-lanina/RL4SE.

Problem

• Survey RL applications in software engineering comprehensively
• Analyze trends and categorize SE topics and RL algorithms
• Identify challenges and future directions for RL in SE

Innovation

• Surveyed 115 RL-for-SE studies systematically
• Integrated RL with LLMs for adaptability
• Analyzed trends, algorithms, and evaluation practices