Leveraging Data-Driven Infrastructure Management to Facilitate AIOps for Big Data Applications and Operations-Reference-Cited by-同舟云学术

Leveraging Data-Driven Infrastructure Management to Facilitate AIOps for Big Data Applications and Operations

Published:2021-07-01 Issue: Volume: Page:135-158
ISSN:
Container-title:Technologies and Applications for Big Data Value
language:
Short-container-title:

Author:

McCreadie Richard,Soldatos John,Fuerst Jonathan,Argerich Mauricio Fadel,Kousiouris George,Totow Jean-Didier,Nieto Antonio Castillo,Navidad Bernat Quesada,Kyriazis Dimosthenis,Macdonald Craig,Ounis Iadh

Abstract

AbstractAs institutions increasingly shift to distributed and containerized application deployments on remote heterogeneous cloud/cluster infrastructures, the cost and difficulty of efficiently managing and maintaining data-intensive applications have risen. A new emerging solution to this issue is Data-Driven Infrastructure Management (DDIM), where the decisions regarding the management of resources are taken based on data aspects and operations (both on the infrastructure and on the application levels). This chapter will introduce readers to the core concepts underpinning DDIM, based on experience gained from development of the Kubernetes-based BigDataStack DDIM platform (https://bigdatastack.eu/). This chapter involves multiple important BDV topics, including development, deployment, and operations for cluster/cloud-based big data applications, as well as data-driven analytics and artificial intelligence for smart automated infrastructure self-management. Readers will gain important insights into how next-generation DDIM platforms function, as well as how they can be used in practical deployments to improve quality of service for Big Data Applications.This chapter relates to the technical priority Data Processing Architectures of the European Big Data Value Strategic Research & Innovation Agenda [33], as well as the Data Processing Architectures horizontal and Engineering and DevOps for building Big Data Value vertical concerns. The chapter relates to the Reasoning and Decision Making cross-sectorial technology enablers of the AI, Data and Robotics Strategic Research, Innovation & Deployment Agenda [34].

Publisher

Springer International Publishing

Link

https://link.springer.com/content/pdf/10.1007/978-3-030-78307-5_7

Reference34 articles.

1. Argerich, M. F., Cheng, B., & Fürst, J. (2019). Reinforcement learning based orchestration for elastic services. In Proceedings of the 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), IEEE (pp. 352–357).

2. Canfora, G., Di Penta, M., Esposito, R., & Villani, M. L. (2005). An approach for qos-aware service composition based on genetic algorithms. In Proceedings of the 7th annual conference on Genetic and evolutionary computation (pp. 1069–1075).

3. Chung, A., Park, J. W., & Ganger, G. R. (2018). Stratus: Cost-aware container scheduling in the public cloud. In Proceedings of the ACM symposium on cloud computing (pp. 121–134).

4. Demchenko, Y., Filiposka, S., Tuminauskas, R., Mishev, A., Baumann, K., Regvart, D., & Breach, T. (2015). Enabling automated network services provisioning for cloud based applications using zero touch provisioning. In 2015 IEEE/ACM 8th international conference on utility and cloud computing (UCC). IEEE (pp. 458–464).

5. Eramo, V., Cianfrani, A., Catena, T., Polverini, M., & Lavacca, F. (2019). Reconfiguration of cloud and bandwidth resources in NFV architectures based on segment routing control/data plane. In Proceedings of the 2019 21st international conference on transparent optical networks (ICTON), IEEE (pp. 1–5).