Ordered Field Property for Semi-Markov Games when One Player Controls Transition Probabilities and Transition Times

Abstract

Two-person finite semi-Markov games (SMGs) are studied when the transition probabilities and the transition times are controlled by one player at all states. For the discounted games in this class, we prove that the ordered field property holds and there exist optimal/Nash equilibrium stationary strategies for the players. We illustrate that the zero-sum SMGs where only transition probabilities are controlled by one player, do not necessarily satisfy the ordered field property. An algorithm along with a numerical example for the discounted one player control zero-sum SMGs is given via linear programming. For the undiscounted version of such games, we exhibit with an example that if the game ceases to be unichain, an optimal stationary or Markov strategy need not exist, (though in this example of a one-player game we exhibit a semi-stationary optimal strategy/policy). Lastly, we prove that if such games are unichain, then they possess the ordered field property for the undiscounted case as well.