Generalizations of Wald's fundamental identity of sequential analysis to Markov chains

Abstract

ABSTRACTAn extension of Bellman's analogue of Wald's identity is proved for Markov processes with a finite number of states, in which (a) the diagonal matrices of the state functions and the transition matrices may be heterogeneous but are so related that the product of the two matrices which apply at any stage has at least one modal vector in common with all of the corresponding products which apply at other stages, and (b) the probability distribution of the duration of any one realization of the process satisfies certain conditions. The condition (a) is called equimodality. The conditions for (b) are satisfied when there is asymptotically a sufficiently abrupt exponential upper bound to the probability distribution of the duration, and extends simply to validate analogues involving latent roots other than the one which appears in Bellman's formula. A certain generalization of Wald's stopping rule produces a bound of this kind.