🤖 AI Summary
This work addresses the lack of a well-defined, actionable metric for assessing the clarity of code review comments (CRCs). We propose RIE—the first industrially grounded, three-dimensional clarity model comprising Relevance, Informativeness, and Expressiveness—and introduce ClearCRC, an automated evaluation framework. Through a systematic literature review, a developer survey (N=217), and empirical analysis across nine programming languages in open-source projects, we find that 28.8% of CRCs exhibit clarity deficiencies in at least one RIE dimension. ClearCRC integrates rule-based and heuristic features, significantly outperforming baseline methods in accuracy and F1-score. This study is the first to systematically characterize developers’ real-world expectations regarding CRC clarity, thereby establishing both a theoretical foundation and a practical toolset for improving code review quality.
📝 Abstract
The code review comment (CRC) is pivotal in the process of modern code review. It provides reviewers with the opportunity to identify potential bugs, offer constructive feedback, and suggest improvements. Clear and concise code review comments (CRCs) facilitate the communication between developers and is crucial to the correct understanding of the issues identified and proposed solutions. Despite the importance of CRCs' clarity, there is still a lack of guidelines on what constitutes a good clarity and how to evaluate it. In this paper, we conduct a comprehensive study on understanding and evaluating the clarity of CRCs. We first derive a set of attributes related to the clarity of CRCs, namely RIE attributes (i.e., Relevance, Informativeness, and Expression), as well as their corresponding evaluation criteria based on our literature review and survey with practitioners. We then investigate the clarity of CRCs in open-source projects written in nine programming languages and find that a large portion (i.e., 28.8%) of the CRCs lack the clarity in at least one of the attributes. Finally, we propose ClearCRC, an automated framework that evaluates the clarity of CRCs. Experimental results show that ClearCRC can effectively evaluate the clarity of CRCs and outperform the baselines.