Value iteration in a class of average controlled Markov chains with unbounded costs: necessary and sufficient conditions for pointwise convergence

Abstract

This work concerns controlled Markov chains with denumerable state space, (possibly)unboundedcost function, and an expectedaveragecost criterion. Under aLyapunov function condition, together with mild continuity-compactness assumptions, a simplenecessary and sufficientcriterion is given so that the relative value functions and differential costs produced by thevalue iterationscheme converge pointwise to the solution of the optimality equation; this criterion is applied to obtain convergence results when the cost function is bounded below or bounded above.