Asymptotic Optimality of Power-of-d Load Balancing in Large-Scale Systems

Abstract

We consider a system of N identical server pools and a single dispatcher in which tasks with unit-exponential service requirements arrive at rate [Formula: see text]. In order to optimize the experienced performance, the dispatcher aims to evenly distribute the tasks across the various server pools. Specifically, when a task arrives, the dispatcher assigns it to the server pool with the minimum number of tasks among d(N) randomly selected server pools. We construct a stochastic coupling to bound the difference in the system occupancy processes between the join-the-shortest-queue (JSQ) policy and a scheme with an arbitrary value of d(N). We use the coupling to derive the fluid limit in case [Formula: see text] and [Formula: see text] as [Formula: see text] along with the associated fixed point. The fluid limit turns out to be insensitive to the exact growth rate of d(N) and coincides with that for the JSQ policy. We further establish that the diffusion limit corresponds to that for the JSQ policy as well, as long as [Formula: see text], and characterize the common limiting diffusion process. These results indicate that the JSQ optimality can be preserved at the fluid and diffusion levels while reducing the overhead by nearly a factor O(N) and O([Formula: see text]), respectively.