Affiliation:
1. Computer Science Department, Technion, Israel
2. Dell EMC, USA
3. ORT Braude College of Engineering, Carmiel, Israel
Abstract
Deduplication reduces the size of the data stored in large-scale storage systems by replacing duplicate data blocks with references to their unique copies. This creates dependencies between files that contain similar content and complicates the management of data in the system. In this article, we address the problem of data migration, in which files are remapped between different volumes as a result of system expansion or maintenance. The challenge of determining which files and blocks to migrate has been studied extensively for systems without deduplication. In the context of deduplicated storage, however, only simplified migration scenarios have been considered.
In this article, we formulate the general migration problem for deduplicated systems as an optimization problem whose objective is to minimize the system’s size while ensuring that the storage load is evenly distributed between the system’s volumes and that the network traffic required for the migration does not exceed its allocation.
We then present three algorithms for generating effective migration plans, each based on a different approach and representing a different trade-off between computation time and migration efficiency. Our
greedy algorithm
provides modest space savings but is appealing thanks to its exceptionally short runtime. Its results can be improved by using larger system representations. Our
theoretically optimal algorithm
formulates the migration problem as an integer linear programming (ILP) instance. Its migration plans consistently result in smaller and more balanced systems than those of the greedy approach, although its runtime is long and, as a result, the theoretical optimum is not always found. Our
clustering algorithm
enjoys the best of both worlds: its migration plans are comparable to those generated by the ILP-based algorithm, but its runtime is shorter, sometimes by an order of magnitude. It can be further accelerated at a modest cost in the quality of its results.
Funder
Israel Science Foundation
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture
Reference59 articles.
1. Cluster analysis. (n.d.). Retrieved October 24 2020 from https://en.wikipedia.org/wiki/Cluster_analysis.
2. CPLEX Optimizer. (n.d.). Retrieved October 24 2018 from https://www.ibm.com/analytics/cplex-optimizer.
3. The Fastest Mathematical Programming Solver. (n.d.). Retrieved October 24 2018 from http://www.gurobi.com/.
4. GLPK (GNU Linear Programming Kit). (n.d.). Retrieved October 24 2018 from https://www.gnu.org/software/glpk/.
5. Introduction to lp_solve 5.5.2.5. (n.d.). Retrieved October 24 2018 from http://lpsolve.sourceforge.net/5.5/.
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献