Affiliation:
1. KAIST, Daejeon, Republic of Korea
2. Hyundai MOBIS, Seoul, Republic of Korea
3. Handong Global University, Pohang, Republic of Korea
Abstract
Mutation analysis can provide valuable insights into both the system under test and its test suite. However, it is not scalable due to the cost of building and testing a large number of mutants. Predictive Mutation Testing (PMT) has been proposed to reduce the cost of mutation testing, but it can only provide statistical inference about whether a mutant will be killed or not by the entire test suite. We propose Seshat, a Predictive Mutation Analysis (PMA) technique that can accurately predict the
entire kill matrix
, not just the Mutation Score (MS) of the given test suite. Seshat exploits the natural language channel in code, and learns the relationship between the syntactic and semantic concepts of each test case and the mutants it can kill, from a given kill matrix. The learnt model can later be used to predict the kill matrices for subsequent versions of the program, even after both the source and test code have changed significantly. Empirical evaluation using the programs in
Defects4J
shows that Seshat can predict kill matrices with an average F-score of 0.83 for versions that are up to years apart. This is an improvement in F-score by 0.14 and 0.45 points over the state-of-the-art PMT technique and a simple coverage-based heuristic, respectively. Seshat also performs as well as PMT for the prediction of the MS only.
When applied to a mutant-based fault localisation technique, the predicted kill matrix by Seshat is successfully used to locate faults within the top 10 position, showing its usefulness beyond prediction of MS.
Once Seshat trains its model using a concrete mutation analysis, the subsequent predictions made by Seshat are on average 39 times faster than actual test-based analysis.
We also show that Seshat can be successfully applied to automatically generated test cases with an experiment using EvoSuite.
Funder
National Research Foundation of Korea
Institute for Information & Communications Technology Promotion
Samsung Electronics
Publisher
Association for Computing Machinery (ACM)
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. FrMi: Fault-revealing Mutant Identification using killability severity;Information and Software Technology;2023-12
2. Contextual Predictive Mutation Testing;Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2023-11-30
3. Predicting higher order mutation score based on machine learning;Journal of Information and Telecommunication;2023-08-31
4. A Systematic Literature Review on Solutions of Mutation Testing Problems;2023 IEEE 8th International Conference On Software Engineering and Computer Systems (ICSECS);2023-08-25