Stochastic orders in partition and random testing of software

Abstract

Testing in order to produce software of high reliability is an area of major concern in software engineering. In an effort to find efficient methods of testing, the comparison of partition and random sampling testing methods has received considerable attention in the literature. A standard criterion for comparisons between random and partition testing, based on their expected efficacy in program debugging, is the probability of detecting at least one failure causing input in the program's domain. However, the goal in software testing is usually to find as many faults as possible in a reasonable period of time, and therefore stochastic comparisons of the number of faults obtained in partition and random testing may provide more valuable information on which testing procedures to use. We establish various conditions which guarantee that the number of faults discovered in partition testing is stochastically greater than the number discovered in random testing (using a fixed total sample size) for many of the well-established stochastic orders (including the usual stochastic order, the hazard rate order, the likelihood ratio order, and the variability order). The results established also allow us to obtain both upper and lower bounds with these stochastic orders for the sum of n independent Bernoulli random trials (with varying probability of success) in terms of the binomial distribution with parameters n and p.