Combinatorial Test Design Using Design-Time Decisions for Variability

Abstract

Design-time artifacts such as module structures from the module view, component diagrams from the component-and-connector view, sequence diagrams, communication diagrams, activity diagrams, or state-machine diagrams from the behavioral view of software architecture can be used to generate effective test cases for fault detection at integration testing level. Various combinatorial test design (CTD) methods have been developed for generating valid and representative product configurations for a software product line, but these methods assume that all bindings for variabilities are performed at run time. Thus, these methods have a limitation in that they do not consider design decisions for variability represented in design-time artifacts. This paper aims to propose a CTD method that uses design-time variability, called the DesignTimeCTD method, considering variability-related decisions at design stage and investigating the effectiveness of reducing the number of tests. The DesignTimeCTD uses variability model, variation points in design-time artifacts, and variability implementation mechanisms used for implementation together in order to reduce the number of tests. As a result, the CTD method, which uses design-time decisions for variability, contributes to reducing the test case by approximately 14% by allowing test engineers to consider design-time variability-related decisions together.