On The Impact of Merge Request Deviations on Code Review Practices

2025-06-10

AI Summary

In industrial practice, numerous merge requests (MRs) deviate from standard code review workflows—e.g., drafts, force-pushes, or dependency updates—distorting review analytics and degrading machine learning model performance. To address this, we propose the first taxonomy of MR deviations, encompassing seven distinct categories, and introduce a few-shot learning–based method for automatic deviation detection. Empirical evaluation on industrial data reveals that deviations constitute 37.02% of MRs and are detected with 91% accuracy. Excluding such deviations improves merge review time prediction in 53.33% of cases—with gains up to 2.25×—and reduces feature importance distribution shift by 47%, substantially enhancing model reliability. This work systematically characterizes the impact of MR workflow heterogeneity on software engineering analytics, establishing both a methodological foundation and empirical evidence for robust review modeling.

Technology Category

• Machine Learning: Evaluation and Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Code review is a key practice in software engineering, ensuring quality and collaboration. However, industrial Merge Request (MR) workflows often deviate from standardized review processes, with many MRs serving non-review purposes (e.g., drafts, rebases, or dependency updates). We term these cases deviations and hypothesize that ignoring them biases analytics and undermines ML models for review analysis. We identify seven deviation categories, occurring in 37.02% of MRs, and propose a few-shot learning detection method (91% accuracy). By excluding deviations, ML models predicting review completion time improve performance in 53.33% of cases (up to 2.25x) and exhibit significant shifts in feature importance (47% overall, 60% top-*k*). Our contributions include: (1) a taxonomy of MR deviations, (2) an AI-driven detection approach, and (3) empirical evidence of their impact on ML-based review analytics. This work aids practitioners in optimizing review efforts and ensuring reliable insights.

Problem

• Identifies deviations in Merge Request workflows affecting code review
• Proposes AI method to detect non-review Merge Request purposes
• Demonstrates impact of deviations on ML-based review analytics

Innovation

• Few-shot learning detects MR deviations
• Excluding deviations improves ML model performance
• Taxonomy and AI approach for MR analysis