Test suite assessment of safety-critical systems using safety tactics and fault-based mutation testing

Abstract

AbstractA safety-critical system is a system in which the software malfunctioning could result in death, injury, or damage to the environment. Addressing safety concerns early on at the architecture design level is critical to guide the subsequent life cycle activities to ensure that the eventual system is reliable. A fundamental approach to address safety at the design level is the adoption of architectural tactics. It is crucial for safety-critical systems to correctly implement the constraints as defined by the selected safety tactics. Given the limitations of traditional safety measures, particularly in conducting thorough analyses of safety tactics at the architectural design level, our motivation is to close this gap by integrating safety tactics and fault-based mutation testing. This article proposes a systematic approach for assessing the adequacy of test suites of safety-critical systems based on these architectural safety tactics. Our proposed approach involves the integration of safety tactics and fault-based mutation testing to comprehensively assess the adequacy of test suites in safety-critical systems. At the core of our methodology is the adoption of architectural safety tactics, addressing potential issues at the design level. We leverage fault-based testing, utilizing a dedicated metamodel and domain-specific language (DSL) to model safety views and their relation to the code. We use a case study to evaluate the effectiveness of our approach using fault-injection techniques. Our study shows that this systematic approach is feasible and effective for test suite assessment of safety-critical systems.