Code Review as Decision-Making -- Building a Cognitive Model from the Questions Asked During Code Review

2025-07-13

AI Summary

While code review enhances software quality and collaboration, existing tools often frustrate developers, and fully automated approaches undermine core socio-cognitive benefits—particularly knowledge transfer. Method: This paper introduces the first systematic Cognitive Review Decision Model (CRDM), conceptualizing code review as a directed–analytical two-stage decision process that captures developers’ recurrent judgment mechanisms. Grounded in 34 ethnographic think-aloud sessions, the model is validated through integrated thematic, temporal, and sequential analysis. Contribution/Results: CRDM demonstrates strong empirical alignment with observed, recognition-driven decision patterns. The study publicly releases a curated annotated dataset and open-source analytical tools, providing both an empirical foundation and a theoretical framework to guide the design of next-generation review support tools—tools that simultaneously optimize efficiency, reviewer satisfaction, and interpersonal value.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Code review is a well-established and valued practice in the software engineering community contributing to both code quality and interpersonal benefits. However, there are challenges in both tools and processes that give rise to misalignments and frustrations. Recent research seeks to address this by automating code review entirely, but we believe that this risks losing the majority of the interpersonal benefits such as knowledge transfer and shared ownership. We believe that by better understanding the cognitive processes involved in code review, it would be possible to improve tool support, with out without AI, and make code review both more efficient, more enjoyable, while increasing or maintaining all of its benefits. In this paper, we conduct an ethnographic think-aloud study involving 10 participants and 34 code reviews. We build a cognitive model of code review bottom up through thematic, statistical, temporal, and sequential analysis of the transcribed material. Through the data, the similarities between the cognitive process in code review and decision-making processes, especially recognition-primed decision-making, become apparent. The result is the Code Review as Decision-Making (CRDM) model that shows how the developers move through two phases during the code review; first an orientation phase to establish context and rationale and then an analytical phase to understand, assess, and plan the rest of the review. Throughout the process several decisions must be taken, on writing comments, finding more information, voting, running the code locally, verifying continuous integration results, etc. Analysis software and process-coded data publicly available at: https://doi.org/10.5281/zenodo.15758266

Problem

• Understanding cognitive processes in code review
• Improving tool support for code review efficiency
• Balancing automation with interpersonal benefits in reviews

Innovation

• Building cognitive model from code review questions
• Ethnographic think-aloud study with participants
• CRDM model with orientation and analytical phases