On the role of compression in distributed systems

Abstract

Compression has been used in numerous ways for many years, but recently two factors have combined in a way to push compression to the forefront of distributed systems. First, the disparity between processor speeds and I/O rates is ever-increasing, making it possible to perform compression in software to a much greater extent than was previously feasible. Second, the growth of new applications demanding enormous data rates, such as digital video and audio, makes hardware compression increasingly desirable: I discuss the importance of compression in various environments and describe how compression may be used not only to reduce the demand for disk space, disk bandwidth, and network bandwidth, but also to appear to extend physical memory.