Abstract
AbstractMany modern applications, both scientific and commercial, are deployed to cloud environments and often employ multiple types of resources. That allows them to efficiently allocate only the resources which are actually needed to achieve their goals. However, in many workloads the actual usage of the infrastructure varies over time, which results in over-provisioning and unnecessarily high costs. In such cases, automatic resource scaling can provide significant cost savings by provisioning only the amount of resources which are necessary to support the current workload. Unfortunately, due to the complex nature of distributed systems, automatic scaling remains a challenge. Reinforcement learning domain has been recently a very active field of research. Thanks to combining it with Deep Learning, many newly designed algorithms improve the state of the art in many complex domains. In this paper we present the results of our attempt to use the recent advancements in Reinforcement Learning to optimize the cost of running a compute-intensive evolutionary process by automating the scaling of heterogeneous resources in a compute cloud environment. We describe the architecture of our system and present evaluation results. The experiments include autonomous management of a sample workload and a comparison of its performance to the traditional automatic threshold-based management approach. We also provide the details of training of the management policy using the proximal policy optimization algorithm. Finally, we discuss the feasibility to extend the presented approach to further scenarios.
Funder
Ministerstwo Edukacji i Nauki
Infrastruktura PL-Grid
Publisher
Springer Science and Business Media LLC
Subject
Hardware and Architecture,Information Systems,Theoretical Computer Science,Software
Reference66 articles.
1. Chen T, Bahsoon R, Yao X (2018) A survey and taxonomy of self-aware and self-adaptive cloud autoscaling systems. ACM Comput Surv 51(3):61–16140
2. Sutton RS (1984) Temporal credit assignment in reinforcement learning. PhD thesis, University of Massachusetts Amherst
3. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. CoRR arXiv:cs.AI/9605103
4. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
5. Gu S, Holly E, Lillicrap T, Levine S (2017) Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389–3396. IEEE International Conference on Robotics and Automation (ICRA), Washington, DC, USA. https://doi.org/10.1109/ICRA.2017.7989385
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献