A study of practical deduplication

Author:

Meyer Dutch T.1,Bolosky William J.2

Affiliation:

1. The University of British Columbia, Microsoft Research

2. Microsoft Research

Abstract

We collected file system content data from 857 desktop computers at Microsoft over a span of 4 weeks. We analyzed the data to determine the relative efficacy of data deduplication, particularly considering whole-file versus block-level elimination of redundancy. We found that whole-file deduplication achieves about three quarters of the space savings of the most aggressive block-level deduplication for storage of live file systems, and 87% of the savings for backup images. We also studied file fragmentation, finding that it is not prevalent, and updated prior file system metadata studies, finding that the distribution of file sizes continues to skew toward very large unstructured files.

Publisher

Association for Computing Machinery (ACM)

Subject

Hardware and Architecture

Reference33 articles.

1. BackupRead. 2010. Microsoft Corp. BackupRead function. MSDN. http://msdn.microsoft.com/en-us/library/aa362509(VS.85).aspx BackupRead. 2010. Microsoft Corp. BackupRead function. MSDN. http://msdn.microsoft.com/en-us/library/aa362509(VS.85).aspx

2. Space/time trade-offs in hash coding with allowable errors

Cited by 225 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Finite State Automata Based Cryptosystem for Secure Data Sharing and De-duplication in Cloud Computing;SN Computer Science;2024-08-08

2. Is Low Similarity Threshold A Bad Idea in Delta Compression?;Proceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems;2024-07-08

3. Convergent encryption enabled secure data deduplication algorithm for cloud environment;Concurrency and Computation: Practice and Experience;2024-06-21

4. A randomized encryption deduplication method against frequency attack;Journal of Information Security and Applications;2024-06

5. Space-efficient and high-performance inline deduplication for emerging hybrid storage system with Libra+;Journal of Systems Architecture;2024-05

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3