Abstract
AbstractIn the era of global-scale services, organisations produce huge volumes of data, often distributed across multiple data centres, separated by vast geographical distances. While cluster computing applications, such as MapReduce and Spark, have been widely deployed in data centres to support commercial applications and scientific research, they are not designed for running jobs across geo-distributed data centres. The necessity to utilise such infrastructure introduces new challenges in the data analytics process due to bandwidth limitations of the inter-data-centre communication. In this article, we discuss challenges and survey the latest geo-distributed big-data analytics frameworks and schedulers (based on MapReduce and Spark) with WAN-bandwidth awareness.
Publisher
Springer Science and Business Media LLC
Subject
Information Systems and Management,Computer Networks and Communications,Hardware and Architecture,Information Systems
Reference74 articles.
1. Tudoran R, Antoniu G, Bougé L. SAGE: geo-distributed streaming data analysis in clouds. In: 2013 IEEE
international symposium on parallel distributed processing, workshops and Phd Forum. 2013, vol. 2013, p. 2278–81.
2. Tudoran R, Costan A, Wang R, Bougé L, Bridging Antoniu G. Data in the clouds: an environment-aware system for geographically distributed data transfers. In: 2014 14th IEEE/ACM international symposium on cluster, cloud and grid computing; 2014, p. 92–101.
3. Cardosa M, Wang C, Nangia A, Chandra A, Weissman J. Exploring mapreduce efficiency with highly-distributed data. In: Proceedings of the second international workshop on mapreduce and its applications. ACM; 2011. p. 27–34.
4. Heintz B, Chandra A, Sitaraman RK, Weissman J. End-to-end optimization for geo-distributed mapreduce. IEEE Trans Cloud Comput. 2016;4(3):293–306.
5. Rabkin A, Arye M, Sen S, Pai V, Freedman MJ. Making every bit count in wide-area analytics. In: Presented as part of the 14th workshop on hot topics in operating systems. USENIX; 2013.
Cited by
10 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献