🤖 AI Summary
This paper addresses key challenges in reviewer recommendation for code review—low accuracy, high latency, imbalanced workload distribution, and pronounced bystander effect—by proposing an industrial-grade optimization framework. Methodologically, it introduces three core innovations: (1) modeling author–reviewer familiarity as a critical feature for the first time; (2) designing a workload-aware re-ranking mechanism that normalizes multi-dimensional reviewer load; and (3) incorporating a stochastic primary-responsibility assignment strategy to mitigate responsibility diffusion. Built upon iterative refinement of the production model RevRecV1, the framework holistically balances accuracy, timeliness, and fairness. Experimental results demonstrate a 14-percentage-point improvement in recommendation accuracy, an 11% reduction in average review completion time, significantly more balanced reviewer workload distribution, and a marked increase in recommendation adoption rate.
📝 Abstract
Code review ensures that a peer engineer manually examines the code before it is integrated and released into production. At Meta, we develop a wide range of software at scale, from social networking to software development infrastructure, such as calendar and meeting tools to continuous integration. We are constantly improving our code review system, and in this work we describe a series of experiments that were conducted across 10's of thousands of engineers and 100's of thousands of reviews. We build upon the recommender that has been in production since 2018, RevRecV1. We found that reviewers were being assigned based on prior authorship of files. We reviewed the literature for successful features and experimented with them with RevRecV2 in production. The most important feature in our new model was the familiarity of the author and reviewer, we saw an overall improvement in accuracy of 14 percentage points. Prior research has shown that reviewer workload is skewed. To balance workload, we divide the reviewer score from RevRecV2 by each candidate reviewers workload. We experimented with multiple types of workload to develop RevRecWL. We find that reranking candidate reviewers by workload often leads to a reviewers with lower workload being selected by authors. The bystander effect can occur when a team of reviewers is assigned the review. We mitigate the bystander effect by randomly assigning one of the recommended reviewers. Having an individual who is responsible for the review, reduces the time take for reviews by -11%.
