Affiliation:
1. University of Wisconsin-Madison, Madison, WI
2. NetApp, Sunnyvale, CA
3. University of Toronto, Toronto, ON
Abstract
An important threat to reliable storage of data is silent data corruption. In order to develop suitable protection mechanisms against data corruption, it is essential to understand its characteristics. In this article, we present the first large-scale study of data corruption. We analyze corruption instances recorded in production storage systems containing a total of 1.53 million disk drives, over a period of 41 months. We study three classes of corruption: checksum mismatches, identity discrepancies, and parity inconsistencies. We focus on checksum mismatches since they occur the most.
We find more than 400,000 instances of checksum mismatches over the 41-month period. We find many interesting trends among these instances, including: (i) nearline disks (and their adapters) develop checksum mismatches an order of magnitude more often than enterprise-class disk drives, (ii) checksum mismatches within the same disk are not independent events and they show high spatial and temporal locality, and (iii) checksum mismatches across different disks in the same storage system are not independent. We use our observations to derive lessons for corruption-proof system design.
Funder
Division of Computing and Communication Foundations
National Science Foundation
Division of Computer and Network Systems
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture
Cited by
76 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. Understanding Silent Data Corruptions in a Large Production CPU Population;Proceedings of the 29th Symposium on Operating Systems Principles;2023-10-23
2. LOGI: an empirical model of heat-induced disk drive data loss and its implications for data recovery;Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering;2022-11-07
3. In-Network Caching Assisted Error Recovery For File Transfers;2022 IEEE/ACM International Workshop on Innovating the Network for Data-Intensive Science (INDIS);2022-11
4. Be SMART, Save I/O: A Probabilistic Approach to Avoid Uncorrectable Errors in Storage Systems;2022 IEEE International Conference on Cluster Computing (CLUSTER);2022-09
5. Secure storage—Confidentiality and authentication;Computer Science Review;2022-05