Discrete time methods for simulating continuous time Markov chains

Abstract

This paper discusses several problems which arise when the regenerative method is used to analyse simulations of Markov chains. The first problem involves calculating the variance constant which appears in the central limit theorem used to obtain confidence intervals. Knowledge of this constant is very helpful in evaluating simulation methodologies. The second problem is to devise a method for simulating continuous time Markov chains without having to generate exponentially distributed holding times. Several methods are presented and compared. Numerical examples are given to illustrate the computional and statistical efficiency of these methods.