Towards Better Crowdsourced Software Testing Process

Abstract

Crowdsourced software testing (CST) is an emerging trend in software testing. Companies and developers assign testing tasks through CST platforms to thousands of online testers. Currently, CST platform managers are trying to find and resolve challenges in order to reach the best CST practice. Many features have been applied by CST platforms to improve CST activities, including notification emails, online chat rooms, forums and, most importantly, a CST platform dashboard to view all testing projects and tasks; these features have enabled CST to operate efficiently. Still, CST users find it difficult to stay abreast of test project updates, maintain their motivation, and avoid frustration. This aligns with the increasing studies in the literature that call for more efficient solutions to support CST process. This research aims to support CST by searching for potential process limitations and overcoming them. In order to do so, a five-stage approach is used: first, the current process of CST is investigated through reviewing 15 CST platforms. Second, the review has resulted in identifying eight possible activities to improve CST; among them six have been selected after a survey distributed to 30 domain experts in Stage 3. In Stage 4, we have designed and implemented five process models in a web-based system that can fulfill the requirements of the six activities identified earlier. In Stage 5, we have evaluated these process models through interviews with representatives of two CST platforms and an expert tester, and through making scenario-based evaluation with 20 domain experts who used the system and rated the value of the processes. The results show that the new improvements are sound and can strengthen the CST practice.