Achieving Tunable Erasure Coding with Cluster-Aware Redundancy Transitioning

Abstract

Erasure coding has been demonstrated as a storage-efficient means against failures, yet its tunability remains a challenging issue in data centers, which is prone to induce substantial cross-cluster traffic. In this article, we present ClusterRT , a cluster-aware redundancy transitioning approach that can dynamically tailor the redundancy degree of erasure coding in data centers. ClusterRT formulates the data relocation as the maximum flow problem to reduce cross-cluster data transfers. It then designs a parity-coordinated update algorithm, which gathers the parity chunks within the same cluster and leverages encoding dependency to further decrease the cross-cluster update traffic. ClusterRT finally rotates the parity chunks to balance the cross-cluster transitioning traffic across the data center. Large-scale simulation and Alibaba Cloud ECS experiments show that ClusterRT reduces 94.0% to 96.2% of transitioning traffic and reduces 70.4% to 88.4% of transitioning time.