Optimal stationary policies for denumerable Markov chains in continuous time

Abstract

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.