Software error analysis

Abstract

The paper reports on a first experimental comparison of software errors generated by real faults and by 1st-order mutations. The experiments were conducted on a program developed by a student from the industrial specification of a critical software from the civil nuclear field. Emphasis was put on the analysis of errors produced upon activation of 12 real faults by focusing on the mechanisms of error creation, masking, and propagation up to failure occurrence, and on the comparison of these errors with those created by 24 mutations. The results involve a total of 3730 errors recorded from program execution traces: 1458 errors were produced by the real faults, and the 2272 others by the mutations. They are in favor of a suitable consistency between errors generated by mutations and by real faults: 85% of the 2272 errors due to the mutations were also produced by the real faults. Moreover, it was observed that although the studied mutations were simple faults, they can create erroneous behaviors as complex as those identified for the real faults. This lends support to the representativeness of errors due to mutations.