A Fault-tolerant Scheduling Strategy through Proactive and Clustering Techniques for Scientific Workflows in Cloud Computing

Abstract

Abstract Scientific workflow scheduling allocates many fine computational granularity tasks to the best appropriate cloud resources. The prevalence of failures in cloud computing is augmented by the substantial quantity of servers and components burdened with resource-intensive workloads. In addition, workflow tasks may face a higher failure risk than a job with the single task. To mitigate the likelihood of these potential failures, the workflow scheduling system should exhibit fault tolerance. In this paper, a fault-tolerant scheduling strategy through proactive and clustering techniques for scientific workflows is proposed in cloud computing. First, the problem of task clustering is formulated by combining several short-duration tasks into a single job to minimize scheduling overhead and enhance the runtime performance of workflow executions. Then, an autonomous framework for workflow scheduling is introduced based on the MAPE-K control model with four essential steps: monitoring, analyzing, planning, and executing, all supported by a shared knowledge base. In the monitoring step, clustered jobs and capabilities of available cloud resources are monitored. In the analyzing step, the failure prediction accuracy is increased by applying the group method of data handling (GMDH) neural network before fault /failure occurrence. In the planning step, (1) the reliability of application execution is assured through a re-clustering technique after fault /failure occurrence; (2) a new hybrid multi-objective algorithm is proposed based on MOPSO and adaptive SA, called MOPSO-aSA, to facilitate workflow scheduling in faulty execution environments. Last, according to the experimental results, it can be concluded that the suggested strategy outperforms other approaches in terms of makespan, total cost, energy consumption, and failure rate.