A Metric for Questions and Discussions Identifying Concerns in Software Reviews

Abstract

Previous studies reported that reviewers ask questions and engage in discussions during software reviews and that the concerns identified by the questions and discussions help detect defects. Although such concerns about potential defects lead to finding defects, review metrics such as the number of defects detected do not always reflect the questions and discussions because concerns which are not applicable to the review material are excluded from the number of defects. This paper proposes a metric, the number of questions and discussions, which identifies concerns in reviews. First, we defined an effective question, which identifies concerns. Then, we defined detailed review processes (identifying, sharing, and recording processes), which capture how concerns identified by effective questions are shared and defects are documented. We conducted a case study with 25 projects in industry to investigate the impact of the number of effective questions, which identified concerns on the number of detected defects in subsequent testing. The results of a multiple regression analysis show that the number of effective questions predicts the number of defects in subsequent testing at the significance level of 0.05.