Conducting Empirical Studies to Evaluate a Technique to Inspect Software Testing Artifacts

Abstract

Experimentation is becoming increasingly used in the Software Engineering field. Several methodologies adopt empirical studies as an instrument to support the software technologies’ conception and maturation. This paper presents the application of a methodology based on the conduction of primary studies to develop a new checklist-based technique, named TestCheck, for inspection of software testing artifacts (test plan, cases, and procedures). Thus far, three empirical studies have been conducted (two feasibility studies and one observational study). The results of the first feasibility study indicate that TestCheck is more efficient and effective when detecting defects in software testing artifacts than an Ad-hoc approach. The results of the second feasibility study suggest an improvement in TestCheck regarding the defects detection and the reduction of the number of false positives per inspector. Finally, the results of the third study (observational study) evaluated positively the application of TestCheck by software engineers inspecting testing artifacts developed in an industrial environment. Thus, it would be qualified for the execution of the methodology’s next steps (case studies in the software industry).