Spark Parallel Optimization Algorithm Based on Improved BP Neural Network

Abstract

Abstract In view of the slow convergence rate and the local minimum problem of BP neural network when it is applied to big data classification, this paper proposes a spark parallel optimization algorithm based on Improved BP neural network. On the basis of MapReduce cloud computing platform, the parallel acceleration is realized by segmenting data sets, and the parallel data is processed by combining the Bagging algorithm, so that the network data of each node can complete the training independently until the convergence is realized, and finally the set becomes the data parallel classifier. The experiment results show that the parallel optimization algorithm proposed in this paper can effectively improve the parallel data processing speed, the classification accuracy is high, and has good adaptability in spark environment.