A Gittins Policy for Optimizing Tail Latency

Abstract

Service level objectives (SLOs) for queueing systems typically relate to the tail of the system's response time distribution T. The tail is the function mapping a time t to the probability P[T > t]. SLOs typically ask that high percentiles of T are not too large, i.e. that P[T > t] is small for large t.