An Analysis on Task Migration Strategy of Big Data Streaming Storm Computing Framework for Distributed Processing

Abstract

In this modern era, a large volume of data is generated regularly, which needs to be processed for gaining profits of latent information. The processing of big data is composed of a challenge termed as communication overhead. In order to minimize communication overhead on the premise of various resource constraints, a task migration strategy under heterogeneous storm environment is proposed. The proposed strategy is based on the establishment and demonstration of storm resource constraint model, optimal communication overhead model, and task migration model. Where the source node selection algorithm adds the nodes beyond the threshold to the source node set according to the load of CPU, memory, and network bandwidth of each working node in the cluster and the priority order of various resources. The experiment shows that compared with the existing research, computing storm can effectively reduce the delay and communication overhead between nodes, and the execution overhead is small.