Approximation theorems for zero-sum nonstationary stochastic games

Abstract

This paper deals with zero-sum nonstationary stochastic games with countable state and action spaces which include both Shapley’s stochastic games [11] and infinite games with imperfect information studied by Orkin in [7]. It is shown that any nonstationary stochastic game with a bounded below lower semicontinuous payoff defined on the space of all histories has a value function and the minimizer has an optimal strategy. Moreover, two approximation theorems extending the main results of Orkin from [7] are established. Finally, counterexamples answering in the negative some open questions raised by Orkin [7] and Sengupta [10] are given.