Abstract
AbstractMutation testing exploits artificial faults to measure the adequacy of test suites and guide their improvement. It has become an extremely popular testing technique as evidenced by the vast literature, numerous tools, and research events on the topic. Previous survey papers have successfully compiled the state of research, its evolution, problems, and challenges. However, the use of mutation testing in practice is still largely unexplored. In this paper, we report the results of a thorough study on the use of mutation testing in GitHub projects. Specifically, we first performed a search for mutation testing tools, 127 in total, and we automatically searched the GitHub repositories including evidence of their use. Then, we focused on the top ten most widely used tools, based on the previous results, and manually revised and classified over 3.5K GitHub active repositories importing them. Among other findings, we observed a recent upturn in interest and activity, with Infection (PHP), PIT (Java) and Humbug (PHP) being the most widely used mutation tools in recent years. The predominant use of mutation testing is development, followed by teaching and learning, and research projects, although with significant differences among mutation tools found in the literature—less adopted and largely used in teaching and research—and those found in GitHub only—more popular and more widely used in development. Our work provides a new and encouraging perspective on the state of practice of mutation testing.
Publisher
Springer Science and Business Media LLC
Cited by
6 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Do we need high-order mutation in fault-based Boolean-specification testing?;Journal of Systems and Software;2024-04
2. An experiment on software test creation: can EvoSuite help test teaching?;Proceedings of the XXII Brazilian Symposium on Software Quality;2023-11-07
3. Do Mutations of Strongly Subsuming Second-Order Mutants Really Mask Each Other?;2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE);2023-10-09
4. Snapshot testing in practice: Benefits and drawbacks;Journal of Systems and Software;2023-10
5. Who Judges the Judge: An Empirical Study on Online Judge Tests;Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis;2023-07-12