Stochastic approximation over multidimensional discrete sets with applications to inventory systems and admission control of queueing networks

Abstract

We propose new methods to solve simulation optimization problems over multidimensional discrete sets. The proposed methods are based on extending the objective function from a discrete domain to a continuous domain and applying stochastic approximation to the extended function. The extension of the objective function is constructed as a piecewise linear interpolation of the original objective function over a particular partition of ℝ d . The advantage of the proposed approach lies in that stochastic approximation is applied to the extension, not the original function, over ℝ d , so the estimated optimal solution at each iteration of the proposed methods is not restricted to be an integer point. Rather, we are free to approach the optimal solution aggressively by moving toward the direction of the steepest descent, thereby skipping over intervening points, thereby resulting in fast convergence in the early stage of the procedures. We provide a set of sufficient conditions under which the proposed methods guarantee the almost sure (a.s.) convergence to the optimal solution. One of such conditions is the multimodularity or L ♮ -convexity of the objective function, which arises in various inventory systems and queueing networks with controlled admission. Numerical examples illustrate the effectiveness of the proposed methods in such settings.