Abstract
We are currently living in the big data era, in which it has become more necessary than ever to develop “smart” schedulers. It is common knowledge that the default Storm scheduler, as well as a large number of static schemes, has presented certain deficiencies. One of the most important of these deficiencies is the weakness in handling cases in which system changes occur. In such a scenario, some type of re-scheduling is necessary to keep the system working in the most efficient way. In this paper, we present a pipeline-based dynamic modular arithmetic-based scheduler (PMOD scheduler), which can be used to re-schedule the streams distributed among a set of nodes and their tasks, when the system parameters (number of tasks, executors or nodes) change. The PMOD scheduler organizes all the required operations in a pipeline scheme, thus reducing the overall processing time.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference26 articles.
1. A review on big data real-time stream processing and its scheduling techniques
2. Linear scheduling of big data streams on multiprocessor sets in the cloud;Tantalaki,2019
3. Apache Stormhttp://storm.apache.org/
4. Spark Streaming-Apache Sparkhttp://spark.apache.org/streaming/
5. Apache Samza—A Distributed Stream Processing Frameworkhttps://samza.apache.org
Cited by
14 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献