Performance Analysis of Several Intelligent Algorithms for Class Integration Test Order Optimization

Abstract

Integration testing is an essential activity in software testing, especially in object-oriented software development. Determining the sequence of classes to be integrated, i.e., the class integration test order (CITO) problem, is of great importance but computationally challenging. Previous research has shown that meta heuristic algorithms can devise class integration test orders with lower test stubbing complexity, resulting in software testing cost reduction. This study focuses on the comparable performance evaluation of ten commonly used meta heuristic algorithms: genetic algorithm (GA), particle swarm optimization (PSO), cuckoo search algorithm (CS), firefly algorithm (FA), bat algorithm (BA), grey wolf algorithm (GWO), moth flame optimization (MFO), sine cosine algorithm (SCA), salp swarm algorithm (SSA) and Harris hawk optimization (HHO). The objective of this study is to identify the most suited algorithms, narrowing down potential avenues for future researches in the field of search-based class integration test order generation. The standard implementations of these algorithms are employed to generate integration test orders. Additionally, these test orders are evaluated and compared in terms of stubbing complexity, convergence speed, average runtime, and memory consumption. The experimental results suggest that MFO, SSA, GWO and CS are the most suited algorithms. MFO, SSA and GWO exhibit excellent optimization performance in systems where fitness values are heavily impacted by attribute coupling. Meanwhile, MFO, GWO and CS are recommended for systems where the fitness values are strongly influenced by method coupling. BA and FA emerge as the slowest algorithms, while the remaining algorithms exhibit intermediate performance. The performance analysis may be used to select and improve appropriate algorithms for the CITO problem, providing a cornerstone for future scientific research and practical applications.