ZERO-KNOWLEDGE BLACKBOX TESTING: WHERE ARE THE FAULTS?

Abstract

We derive a quantitative relationship between the maximal entropy rate achieved by a blackbox software system's specification graph, and the probability of faults Pnobtained by testing the system, as a function of the length n of a test sequence. By equating “blackbox” to the maximal entropy principle, we model the specification graph as a Markov chain that, for each distinct value of n, achieves the maximal entropy rate for that n. Hence the Markov transition probability matrices are not constant in n, but form a sequence of transition matrices T1,…, Tn. We prove that, for nontrivial specification graphs, the probability of finding faults goes asymptotically to zero as the test length n increases, regardless of the evolution of Tn. This implies that zero-knowledge testing is practical only for small n. We illustrate the result using a concrete example of a system specification graph for an autopilot control system, and plot its curve Pn.