Affiliation:
1. Stony Brook University, Stony Brook, NY
2. IBM T. J. Watson Research Center, Yorktown Heights, NY
Abstract
Benchmarking is critical when evaluating performance, but is especially difficult for file and storage systems. Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is rather difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The large variety of workloads that these systems experience in the real world also adds to this difficulty.
In this article we survey 415 file system and storage benchmarks from 106 recent papers. We found that most popular benchmarks are flawed and many research papers do not provide a clear indication of true performance. We provide guidelines that we hope will improve future performance evaluations. To show how some widely used benchmarks can conceal or overemphasize overheads, we conducted a set of experiments. As a specific example, slowing down read operations on ext2 by a factor of 32 resulted in only a 2--5% wall-clock slowdown in a popular compile benchmark. Finally, we discuss future work to improve file system and storage benchmarking.
Funder
Intel Corporation
National Science Foundation
Hewlett-Packard Development Company
Division of Computer and Network Systems
Division of Computing and Communication Foundations
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture
Reference166 articles.
1. Akkerman W. 2002. Strace software home page. www.liacs.nl/~wichert/strace/. Akkerman W. 2002. Strace software home page. www.liacs.nl/~wichert/strace/.
Cited by
68 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献