A Hybrid Scheduler for Many Task Computing in Big Data Systems
Author:
Vasiliu Laura1, Pop Florin12, Negru Catalin1, Mocanu Mariana1, Cristea Valentin1, Kolodziej Joanna3
Affiliation:
1. Computer Science Department, Faculty of Automatic Control and Computers University Politehnica of Bucharest, 313, Splaiul Independentei, 060042 Bucharest , Romania 2. National Institute for Research and Development in Informatics (ICI) 8–10, Mare¸sal Averescu, 011455 Bucharest , Romania 3. Institute of Computer Science Cracow University of Technology, ul. Warszawska 24, 31-155 Cracow , Poland
Abstract
Abstract
With the rapid evolution of the distributed computing world in the last few years, the amount of data created and processed has fast increased to petabytes or even exabytes scale. Such huge data sets need data-intensive computing applications and impose performance requirements to the infrastructures that support them, such as high scalability, storage, fault tolerance but also efficient scheduling algorithms. This paper focuses on providing a hybrid scheduling algorithm for many task computing that addresses big data environments with few penalties, taking into consideration the deadlines and satisfying a data dependent task model. The hybrid solution consists of several heuristics and algorithms (min-min, min-max and earliest deadline first) combined in order to provide a scheduling algorithm that matches our problem. The experimental results are conducted by simulation and prove that the proposed hybrid algorithm behaves very well in terms of meeting deadlines.
Publisher
Walter de Gruyter GmbH
Subject
Applied Mathematics,Engineering (miscellaneous),Computer Science (miscellaneous)
Reference41 articles.
1. Aamodt, K., Quintana, A.A., Achenbach, R., Acounis, S., Adler, C., Aggarwal, M., Agnese, F., Rinella, G.A., Ahammed, Z. and Ahmad, A. (2008). The Alice experiment at the CERN LHC, Journal of Instrumentation 3(08): S08002. 2. Benziani, Y., Kacem, I., Laroche, P. and Nagih, A. (2014). Exact and heuristic methods for minimizing the total completion time in job-shops, Studies in Informatics and Control 23(1): 31-40. 3. Bessis, N., Sotiriadis, S., Cristea, V. and Pop, F. (2011). Modelling requirements for enabling meta-scheduling in inter-clouds and inter-enterprises, 2011 3rd International Conference on Intelligent Networking and Collaborative Systems (INCoS), Fukuoka, Japan, pp. 149-156. 4. Bourdena, A., Mavromoustakis, C. X., Kormentzas, G., Pallis, E., Mastorakis, G. and Yassein, M.B. (2014). A resource intensive traffic-aware scheme using energy-aware routing in cognitive radio networks, Future Generation Computer Systems 39: 16-28. 5. Cabrera, G., Niklander, S., Cabrera, E. and Johnson, F. (2016). Solving a distribution network design problem by means of evolutionary algorithms, Studies in Informatics and Control 25(1): 21-28.
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. Deadline-aware Scheduling in Cloud-Fog-Edge Systems;2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID);2020-05 2. Apache Spark and Apache Ignite Performance Analysis;2019 22nd International Conference on Control Systems and Computer Science (CSCS);2019-05 3. QTM: Computational package using MPI protocol for Quantum Trajectories Method;PLOS ONE;2018-12-10 4. Resource Management System for HPC Computing;Advances in Intelligent Systems and Computing;2018
|
|