Modern code reviews in open-source projects: which problems do they fix?

TLDR

Code review is a manual assessment of source code to detect defects and quality issues, and the lightweight Modern Code Review variant is widely used in industry and open‑source projects. The objective of this paper is to increase our understanding of the practical benefits that the MCR process produces on reviewed source code. The authors empirically explored MCR in OSS by manually classifying over 1,400 reviewed code changes from two projects into a validated scheme and analyzing patterns to uncover technical factors influencing the number of changes. The study found that MCR fixes in OSS mirror industry patterns with a 75:25 maintainability‑to‑functional ratio, that 7–35 % of review comments are discarded and 10–22 % of changes lack explicit comment triggers, that bug‑fixing tasks generate fewer changes while higher file alteration and code churn increase changes, and that reviewer identity does not affect change count.

Abstract

Code review is the manual assessment of source code by humans, mainly intended to identify defects and quality problems. Modern Code Review (MCR), a lightweight variant of the code inspections investigated since the 1970s, prevails today both in industry and open-source software (OSS) systems. The objective of this paper is to increase our understanding of the practical benefits that the MCR process produces on reviewed source code. To that end, we empirically explore the problems fixed through MCR in OSS systems. We manually classified over 1,400 changes taking place in reviewed code from two OSS projects into a validated categorization scheme. Surprisingly, results show that the types of changes due to the MCR process in OSS are strikingly similar to those in the industry and academic systems from literature, featuring the similar 75:25 ratio of maintainability-related to functional problems. We also reveal that 7–35% of review comments are discarded and that 10–22% of the changes are not triggered by an explicit review comment. Patterns emerged in the review data; we investigated them revealing the technical factors that influence the number of changes due to the MCR process. We found that bug-fixing tasks lead to fewer changes and tasks with more altered files and a higher code churn have more changes. Contrary to intuition, the person of the reviewer had no impact on the number of changes.