Numerical methods for infinite Markov processes

Abstract

The estimation of steady state probability distributions of discrete Markov processes with infinite state spaces by numerical methods is investigated. The aim is to find a method applicable to a wide class of problems with a minimum of prior analysis. A general method of numbering discrete states in infinite domains is developed and used to map the discrete state spaces of Markov processes into the positive integers, for the purpose of applying standard numerical techniques. A method based on a little used theoretical result is proposed and is compared with two other algorithms previously used for finite state space Markov processes.