Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

Abstract

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to beunbounded, and the payoff rates may haveneither upper nor lower bounds. We give conditions on the game'sprimitive dataunder which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using theextended infinitesimal operatorassociated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide arecursiveway of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.