Bounds for discounted stochastic scheduling problems

Abstract

Suppose that π is a policy for resource allocation in a stochastic environment andπ ∗is an optimal policy. Two existing procedures for policy evaluation are described and compared. Both of these evaluateπby means of upper bounds onR(π ∗)– R(π), the total reward lost when making resource allocations according toπrather than π∗. The bounds developed by these two methods are called Type 1 and Type 2. We demonstrate by example that neither of these procedures dominates the other in the sense of always yielding tighter bounds. A modification to Type 2 bounds is proposed resulting in an improved procedure which always dominates the Type 1 approach.