A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software Testing

Abstract

Gamification is an established practice in Software Engineering to increase effectiveness and engagement in many practices. This manuscript provides a characterization of the application of gamification to the Software Testing area. Such practice in fact reportedly suffers from low engagement by both personnel in industrial contexts and learners in educational contexts. Our goal is to identify the application areas and utilized gamified techniques and mechanics, the provided benefits and drawbacks, as well as the open challenges in the field. To this purpose, we conducted a Multivocal Literature Review to identify white and grey literature sources addressing gamified software testing. We analyzed 73 contributions and summarized the most common gamified mechanics, concepts, tools, and domains where they are mostly applied. We conclude that gamification in software testing is mostly applied to the test creation phase with simple white-box unit or mutation testing tools and is mostly used to foster good behaviors by promoting the testers’ accomplishment. Key research areas and main challenges in the field are: careful design of tailored gamified mechanics for specific testing techniques; the need for technological improvements to enable crowdsourcing, cooperation, and concurrency; the necessity for empirical and large-scale evaluation of the benefits delivered by gamification mechanics.