Explicit transient probabilities of various Markov models

Abstract

In analyzing finite-state Markov chains knowing the exact eigenvalues of the transition probability matrix P P is important information for predicting the explicit transient behavior of the system. Once the eigenvalues of P P are known, linear algebra and duality theory are used to find P k P^{k} where k = 2 , 3 , 4 , … k= 2,3,4,\ldots . This article is about finding explicit eigenvalue formulas, that scale up with the dimension of P P for various Markov chains. Eigenvalue formulas and expressions of P k P^{k} are first presented when P P is tridiagonal and Toeplitz. These results are generalized to tridiagonal matrices with alternating birth-death probabilities. More general eigenvalue formulas and expression of P k P^{k} are obtained for non-tridiagonal transition matrices P P that have both catastrophe-like and birth-death transitions. Similar results for circulant matrices are also explored. Applications include finding probabilities of sample paths restricted to a strip and generalized ballot box problems. These results generalize to Markov processes with P k P^{k} being replaced by e Q t e^{Qt} where Q Q is a transition rate matrix.