Code Reviews with Divergent Review Scores: An Empirical Study of the OpenStack and Qt Communities

by   Toshiki Hirao, et al.

Code review is a broadly adopted software quality practice where developers critique each others' patches. In addition to providing constructive feedback, reviewers may provide a score to indicate whether the patch should be integrated. Since reviewer opinions may differ, patches can receive both positive and negative scores. If reviews with divergent scores are not carefully resolved, they may contribute to a tense reviewing culture and may slow down integration. In this paper, we study patches with divergent review scores in the OPENSTACK and QT communities. Quantitative analysis indicates that patches with divergent review scores: (1) account for 15 that receive multiple review scores; (2) are integrated more often than they are abandoned; and (3) receive negative scores after positive ones in 70 cases. Furthermore, a qualitative analysis indicates that patches with strongly divergent scores that: (4) are abandoned more often suffer from external issues (e.g., integration planning, content duplication) than patches with weakly divergent scores and patches without divergent scores; and (5) are integrated often address reviewer concerns indirectly (i.e., without changing patches). Our results suggest that review tooling should integrate with release schedules and detect concurrent development of similar patches to optimize review discussions with divergent scores. Moreover, patch authors should note that even the most divisive patches are often integrated through discussion, integration timing, and careful revision.


page 1

page 2


Does Code Review Promote Conformance? A Study of OpenStack Patches

Code Review plays a crucial role in software quality, by allowing review...

Is It Safe to Uplift This Patch? An Empirical Study on Mozilla Firefox

In rapid release development processes, patches that fix critical issues...

An Exploration of Cross-Patch Collaborations via Patch Linkage in OpenStack

Contemporary development projects benefit from code review as it improve...

Automatic Patch Linkage Detection in Code Review Using TextualContent and File Location Features

Context: Contemporary code review tools are a popular choice for softwar...

The Impact of Human Factors on the Participation Decision of Reviewers in Modern Code Review

Modern Code Review (MCR) plays a key role in software quality practices....

Fixing Model Bugs with Natural Language Patches

Current approaches for fixing systematic problems in NLP models (e.g. re...

To Patch, or not To Patch? That is the Question: A Case Study of System Administrators' Online Collaborative Behaviour

System administrators, similar to end users, may delay or avoid software...

Please sign up or login with your details

Forgot password? Click here to reset