Expectations, outcomes, and challenges of modern code review

TLDR

Code review is a widely used practice in both open‑source and industrial settings, now more lightweight than the formal inspections of the 1970s and 1980s. The study empirically investigates the motivations, challenges, and outcomes of tool‑based code reviews and offers recommendations for practitioners and researchers. Researchers observed, interviewed, and surveyed developers and managers, manually classifying hundreds of review comments from diverse Microsoft teams. Although defect detection remains the primary motivation, reviews mainly provide knowledge transfer, team awareness, and alternative solutions, with code and change understanding identified as the key aspect and many needed mechanisms unsupported by current tools.

Abstract

Code review is a common software engineering practice employed both in open source and industrial contexts. Review today is less formal and more “lightweight” than the code inspections performed and studied in the 70s and 80s. We empirically explore the motivations, challenges, and outcomes of tool-based code reviews. We observed, interviewed, and surveyed developers and managers and manually classified hundreds of review comments across diverse teams at Microsoft. Our study reveals that while finding defects remains the main motivation for review, reviews are less about defects than expected and instead provide additional benefits such as knowledge transfer, increased team awareness, and creation of alternative solutions to problems. Moreover, we find that code and change understanding is the key aspect of code reviewing and that developers employ a wide range of mechanisms to meet their understanding needs, most of which are not met by current tools. We provide recommendations for practitioners and researchers.