The Role of Hysteresis in Caching Systems

Abstract

Caching is a fundamental element of networking systems since the early days of the Internet. By filtering requests toward custodians, caches reduce the bandwidth required by the latter and the delay experienced by clients. The requests that are not served by a cache, in turn, comprise its miss stream. We refer to the dependence of the cache state and miss stream on its history as hysteresis. Although hysteresis is at the core of caching systems, a dimension that has not been systematically studied in previous works relates to its impact on caching systems between misses, evictions, and insertions. In this article, we propose novel mechanisms and models to leverage hysteresis on cache evictions and insertions. The proposed solutions extend TTL-like mechanisms and rely on two knobs to tune the time between insertions and evictions given a target hit rate. We show the general benefits of hysteresis and the particular improvement of the two thresholds strategy in reducing download times, making the system more predictable and accounting for different costs associated with object retrieval.