How Do Developers Use Code Suggestions in Pull Request Reviews?

2025-02-07

AI Summary

This study investigates the usage patterns, impact mechanisms, and sociotechnical factors governing code suggestions in GitHub Pull Request (PR) reviews. Addressing a gap in empirical understanding of collaborative code review, we analyze PR metadata from 46 open-source projects, conduct open coding and regression analyses, and supplement findings with a developer survey. Methodologically, we systematically classify suggestions into four empirically grounded categories—stylistic, improvement, bug-fix, and documentation—with “improvement” being the most prevalent. Results show that suggestions significantly increase PR acceptance rates but prolong resolution time; novice contributors receive disproportionately more suggestions; and developers primarily leverage them for traceable justification of changes and learning canonical implementation patterns—highlighting their role as vehicles for tacit knowledge transfer. Crucially, the study reveals how social coding dynamics (e.g., contributor experience, project norms) shape both suggestion initiation and adoption. These findings provide empirically grounded insights for designing intelligent automated review tools and improving open-source collaboration practices.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Interaction Techniques and Devices

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Community question answering

Abstract

GitHub introduced the suggestion feature to enable reviewers to explicitly suggest code modifications in pull requests. These suggestions make the reviewers' feedback more actionable for the submitters and represent a valuable knowledge for newcomers. Still, little is known about how code review suggestions are used by developers, what impact they have on pull requests, and how they are influenced by social coding dynamics. To bridge this knowledge gap, we conducted an empirical study on pull requests from 46 engineered GitHub projects, in which developers used code review suggestions. We applied an open coding approach to uncover the types of suggestions and their usage frequency. We also mined pull request characteristics and assessed the impact of using suggestions on merge rate, resolution time, and code complexity. Furthermore, we conducted a survey with contributors of the studied projects to gain insights about the influence of social factors on the usage and acceptance of code review suggestions. We were able to uncover four suggestion types: code style suggestions, improvements, fixes, and documentation with improvements being the most frequent. We found that the use of suggestions positively affects the merge rate of pull requests but significantly increases resolution time without leading to a decrease in code complexity. Our survey results show that suggestions are more likely to be used by reviewers when the submitter is a newcomer. The results also show that developers mostly search suggestions when tracking rationale or looking for code examples. Our work provides insights on the usage of code suggestions and their potential as a knowledge sharing tool.

Problem

• Analyzing usage patterns of code suggestions in pull requests.
• Assessing impact of suggestions on merge rates and resolution times.
• Exploring social dynamics influencing suggestion acceptance and usage.

Innovation

• Applied open coding approach
• Mined pull request characteristics
• Conducted survey with contributors