Affiliation:
1. School of Computer Science and Engineering Southwest Minzu University Chengdu China
2. Key Laboratory for Computer Systems of State Ethnic Affairs Commission Southwest Minzu University Chengdu China
3. Laboratory for Internet Software Technologies Institute of Software Chinese Academy of Sciences Beijing China
4. School of Computer Science and Technology Hangzhou Dianzi University Hangzhou China
5. Southwest Minzu University Chengdu China
6. School of Electrical Engineering Southwest Jiaotong Univeristy Chengdu China
Abstract
SummaryContinuous integration (CI) is a widely applied development practice to allow frequent integration of software changes, detecting early faults. However, extremely frequent builds consume amounts of time and resources in such a scenario. It is quite challenging for existing test case prioritization (TCP) to address this issue due to the time‐consuming information collection (e.g. test coverage) or inaccurately modelling code semantics to result in the unsatisfied prioritization. In this paper, we propose a semantic‐aware two‐phase TCP framework, named SatTCP, which combines the coarse‐grained filtering and fine‐grained prioritization to perform the precise TCP with low time costs for CI. It consists of three parts: (1) code representation, parsing the programme changes and test cases to obtain the code change and test case representations; (2) coarse‐grained filtering, conducting the preliminary ranking and filtering of test cases based on information retrieval; and (3) fine‐grained prioritization, training a pretrained Siamese language model based on the filtered test set to further sort the test cases via semantic similarity. We evaluate SatTCP on a large‐scale, real‐world dataset with cross‐project validation from fault detection efficiency and time costs and compare it with five baselines. The results show that SatTCP outperforms all baselines by 6.3%–45.6% for mean average percentage of fault detected per cost (APFDc), representing an obvious upward trend as the project scale increases. Meanwhile, SatTCP can reduce the real CI testing by 71.4%, outperforming the best baseline by 17.2% for time costs on average. Furthermore, we discuss the impact of different configurations, flaky tests and hybrid techniques on the performance of SatTCP, respectively.
Funder
National Natural Science Foundation of China
Sichuan Province Science and Technology Support Program
Subject
Safety, Risk, Reliability and Quality,Software