Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning-Reference-Cited by-同舟云学术

Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Published:2024-03-14 Issue: Volume: Page:
ISSN:
Container-title:
language:
Short-container-title:

Author:

Sankaran Lakshmi¹,JS Saleema²,Suleiman Basem³

Affiliation:

1. CMR University

2. Christ University

3. The University of Sydney

Abstract

Abstract With the increasing use of deep reinforcement learning (DRL) techniques to build intelligent systems, the application of it to real-world problems is rampant. Resource allocation in a cloud environment that need dynamic and auto-scaling features is evolving. The agent-based decisions that are offered by DRL are in use by software robotics. Auto-scaling of resources in cloud applications introduces intelligence to agents thus built by these DRL techniques. Markov decision process as a tool minimizes the target rewards to agents such that auto-scaling of applications is performed by agent decisions. Analysis of optimizing the convergence errors that are measured while the agent performs in an online environment is the challenge. Speedy Q-learning (SQL), Generalized SQL(GSQL) algorithm variants relax the parameter values of convergence with a model-free space. The authors applied heuristic values for one such relaxation parameter in our experiments. The study is an extension of works that introduced GSQL-w, where w is the convergence parameter. The authors designed a new GSQL-wh algorithm that heuristically fixes a value for w optimally in cases with over-utilization of resources. This is presented as a novel solution in this study for cloud resource workloads.

Publisher

Research Square Platform LLC

Reference13 articles.

1. D. Edsinger, “Auto-scaling cloud infrastructure with Reinforcement Learning A comparison between multiple RL algorithms to auto-scale resources in cloud infrastructure”, Chalmers University of Technology, Sweden 2018.

2. C. Bitsakos, I. Konstantinou, and N. Koziris, “DERP: A Deep Reinforcement Learning Cloud System for Elastic Resource Provisioning,” in 2018 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), vol. 2018-Decem, Dec 2018, pp. 21–29,10.1109/CloudCom2018.2018.00020.

3. I. John and S. Bhatnagar, “Deep Reinforcement Learning with Successive Over-Relaxation and its Application in Autoscaling Cloud Resources,” International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1–6, 10.1109/IJCNN48605.2020.9206598.

4. P. Singh, P. Gupta, K. Jyoti, and A. Nayyar, “Research on Auto-Scaling of Web Applications in Cloud: Survey, Trends and Future Directions,” Scalable Comput. Pract. Exp., vol. 20, no. 2, pp. 399–432, May 2019,10.12694/scope.v20i2.1537.

5. I. John, C. Kamanchi, and S. Bhatnagar, “Generalized Speedy Q-Learning,” IEEE Control Syst. Lett., vol. 4, no. 3, pp. 524–529, Jul. 2020,10.1109/LCSYS.2020.2970555.